2021-08-11T16:02:29.674Z [main] WARN o.o.s.c.util.YamlToObjectConverter RequestID=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= The file /app/jetty/config/catalog-be/neo4j-errors-configuration.yaml cannot be found. Ignore reading configuration. 2021-08-11T16:02:29.797Z [main] INFO o.o.s.be.listen.BEAppContextListener RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= ASDC version from MANIFEST is 1.9.0 2021-08-11T16:02:29.839Z [main] INFO o.o.s.b.m.BeMonitoringService RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= BE monitoring service is disabled 2021-08-11T16:02:29.840Z [main] INFO o.s.web.context.ContextLoader RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= Root WebApplicationContext: initialization started 2021-08-11T16:02:31.731Z [main] INFO o.o.s.b.d.j.JanusGraphClient RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= ** JanusGraphClient created 2021-08-11T16:02:31.735Z [main] INFO o.o.s.b.d.j.JanusGraphClient RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= ** createGraph started ** 2021-08-11T16:02:31.735Z [main] INFO o.o.s.b.d.j.JanusGraphClient RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= ** open graph with /app/jetty/config/catalog-be/janusgraph.properties started 2021-08-11T16:02:31.735Z [main] INFO o.o.s.b.d.j.JanusGraphClient RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= openGraph : try to load file /app/jetty/config/catalog-be/janusgraph.properties 2021-08-11T16:02:32.022Z [main] INFO com.datastax.driver.core RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= DataStax Java driver 3.8.0 for Apache Cassandra 2021-08-11T16:02:32.028Z [main] INFO c.d.driver.core.GuavaCompatibility RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= Detected Guava >= 19 in the classpath, using modern compatibility layer 2021-08-11T16:02:32.054Z [main] INFO c.d.driver.core.SystemProperties RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= com.datastax.driver.USE_NATIVE_CLOCK is defined, using value false 2021-08-11T16:02:32.054Z [main] INFO c.datastax.driver.core.ClockFactory RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= Using java.lang.System clock to generate timestamps. 2021-08-11T16:02:32.553Z [main] INFO com.datastax.driver.core.NettyUtil RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= Found Netty's native epoll transport in the classpath, using it 2021-08-11T16:02:33.521Z [main] INFO c.d.d.c.p.DCAwareRoundRobinPolicy RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= Using provided data-center name 'Pod' for DCAwareRoundRobinPolicy 2021-08-11T16:02:33.522Z [main] INFO com.datastax.driver.core.Cluster RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= New Cassandra host /10.242.110.137:9042 added 2021-08-11T16:02:33.523Z [main] INFO com.datastax.driver.core.Cluster RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= New Cassandra host /10.242.154.72:9042 added 2021-08-11T16:02:33.523Z [main] INFO com.datastax.driver.core.Cluster RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= New Cassandra host cassandra.onap/10.242.184.131:9042 added 2021-08-11T16:02:33.904Z [main] WARN o.j.g.c.GraphDatabaseConfiguration RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= Local setting storage.cql.replication-strategy-options=[Pod, 3] (Type: FIXED) is overridden by globally managed value ([Pod, 3]). Use the ManagementSystem interface instead of the local configuration to control this setting. 2021-08-11T16:02:35.966Z [main] INFO o.j.g.c.GraphDatabaseConfiguration RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= Generated unique-instance-id=0af218df116-dev-sdc-be-997dd74c7-dmm7p1 2021-08-11T16:02:35.981Z [main] INFO c.d.driver.core.SystemProperties RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= com.datastax.driver.USE_NATIVE_CLOCK is defined, using value false 2021-08-11T16:02:35.981Z [main] INFO c.datastax.driver.core.ClockFactory RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= Using java.lang.System clock to generate timestamps. 2021-08-11T16:02:36.220Z [main] INFO c.d.d.c.p.DCAwareRoundRobinPolicy RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= Using provided data-center name 'Pod' for DCAwareRoundRobinPolicy 2021-08-11T16:02:36.220Z [main] INFO com.datastax.driver.core.Cluster RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= New Cassandra host /10.242.110.137:9042 added 2021-08-11T16:02:36.220Z [main] INFO com.datastax.driver.core.Cluster RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= New Cassandra host /10.242.154.72:9042 added 2021-08-11T16:02:36.220Z [main] INFO com.datastax.driver.core.Cluster RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= New Cassandra host cassandra.onap/10.242.184.131:9042 added 2021-08-11T16:02:36.485Z [main] INFO org.janusgraph.diskstorage.Backend RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= Initiated backend operations thread pool of size 2 2021-08-11T16:02:36.796Z [main] INFO o.j.diskstorage.log.kcvs.KCVSLog RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= Loaded unidentified ReadMarker start time 2021-08-11T16:02:36.796113Z into org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller@5f606ab6 2021-08-11T16:02:36.929Z [main] INFO o.o.s.b.d.j.JanusGraphClient RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= ** JanusGraph graph created 2021-08-11T16:02:38.283Z [main] INFO o.o.s.b.d.j.JanusGraphGenericDao RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= ** JanusGraphGenericDao created 2021-08-11T16:02:38.291Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-08-11T16:02:38.512Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= All sdc keyspaces are : [sdcartifact, sdcrepository, sdcaudit] 2021-08-11T16:02:38.513Z [main] INFO o.o.s.b.d.c.schema.SdcSchemaUtils RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= Connecting to node: [cassandra.onap] port: 9042. 2021-08-11T16:02:38.514Z [main] INFO o.o.s.b.d.c.schema.SdcSchemaUtils RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= Connection timeout in seconds : 120 2021-08-11T16:02:38.514Z [main] INFO o.o.s.b.d.c.schema.SdcSchemaUtils RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= SocketConnectTimeout was provided, setting Cassandra client to use SocketConnectTimeout: 20000 . 2021-08-11T16:02:38.514Z [main] INFO o.o.s.b.d.c.schema.SdcSchemaUtils RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= SocketReadTimeout was provided, setting Cassandra client to use SocketReadTimeout: 20000 . 2021-08-11T16:02:38.514Z [main] INFO o.o.s.b.d.c.schema.SdcSchemaUtils RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= localDatacenter was provided, setting Cassndra clint to use datacenter: Pod as local. 2021-08-11T16:02:38.514Z [main] INFO c.d.driver.core.SystemProperties RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= com.datastax.driver.USE_NATIVE_CLOCK is defined, using value false 2021-08-11T16:02:38.514Z [main] INFO c.datastax.driver.core.ClockFactory RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= Using java.lang.System clock to generate timestamps. 2021-08-11T16:02:38.514Z [main] INFO o.o.s.b.d.c.schema.SdcSchemaUtils RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= ** cluster created 2021-08-11T16:02:38.514Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= creating cluster for Cassandra Health Check. 2021-08-11T16:02:38.903Z [main] INFO c.d.d.c.p.DCAwareRoundRobinPolicy RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= Using provided data-center name 'Pod' for DCAwareRoundRobinPolicy 2021-08-11T16:02:38.904Z [main] INFO com.datastax.driver.core.Cluster RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= New Cassandra host cassandra.onap/10.242.110.137:9042 added 2021-08-11T16:02:38.904Z [main] INFO com.datastax.driver.core.Cluster RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= New Cassandra host cassandra.onap/10.242.154.72:9042 added 2021-08-11T16:02:38.904Z [main] INFO com.datastax.driver.core.Cluster RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= New Cassandra host cassandra.onap/10.242.184.131:9042 added 2021-08-11T16:02:38.904Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= Cluster Metadata: com.datastax.driver.core.Metadata@1a4340f2 2021-08-11T16:02:38.904Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= keyspace : sdcartifact , replication: {Pod=3, class=org.apache.cassandra.locator.NetworkTopologyStrategy} 2021-08-11T16:02:38.904Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= keyspace : sdcrepository , replication: {Pod=3, class=org.apache.cassandra.locator.NetworkTopologyStrategy} 2021-08-11T16:02:38.904Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= keyspace : sdcaudit , replication: {Pod=3, class=org.apache.cassandra.locator.NetworkTopologyStrategy} 2021-08-11T16:02:38.904Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= maxReplication Factor is: 3 2021-08-11T16:02:38.904Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= localQuorum is: 2 2021-08-11T16:02:38.904Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName= TargetServiceName=null ErrorCode=900 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= Health Check formula : Replication Factor – Local_Quorum = 1 2021-08-11T16:02:38.917Z [main] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ECOMP Portal health check configuration is missing. 2021-08-11T16:02:38.928Z [main] INFO o.o.s.b.s.d.SwitchoverDetector RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null Enter init method of SwitchoverDetector 2021-08-11T16:02:38.928Z [main] INFO o.o.s.b.s.d.SwitchoverDetector RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null switchover detector service is disabled 2021-08-11T16:02:39.302Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null creating cluster for Cassandra for monitoring. 2021-08-11T16:02:39.303Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null The cassandra session is com.datastax.driver.core.SessionManager@3bd7829b 2021-08-11T16:02:39.303Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null The number of cassandra nodes is:3 2021-08-11T16:02:39.305Z [main] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null The cassandra down nodes number is 0 2021-08-11T16:02:40.550Z [main] INFO o.o.s.be.dao.jsongraph.JanusGraphDao RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ** JanusGraphDao created 2021-08-11T16:02:40.879Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=070af056-a62d-4095-a9cb-ad52165bfbf2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:02:40.879Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=070af056-a62d-4095-a9cb-ad52165bfbf2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@48896ff7 2021-08-11T16:02:40.879Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=070af056-a62d-4095-a9cb-ad52165bfbf2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:02:40.879Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=070af056-a62d-4095-a9cb-ad52165bfbf2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:02:42.649Z [main] INFO o.o.s.b.d.cassandra.CassandraClient RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null SocketConnectTimeout was provided, setting Cassandra client to use SocketConnectTimeout: 20000 . 2021-08-11T16:02:42.649Z [main] INFO o.o.s.b.d.cassandra.CassandraClient RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null SocketReadTimeout was provided, setting Cassandra client to use SocketReadTimeout: 20000 . 2021-08-11T16:02:42.649Z [main] INFO o.o.s.b.d.cassandra.CassandraClient RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null localDatacenter was provided, setting Cassndra clint to use datacenter: Pod as local. 2021-08-11T16:02:42.649Z [main] INFO c.d.driver.core.SystemProperties RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null com.datastax.driver.USE_NATIVE_CLOCK is defined, using value false 2021-08-11T16:02:42.649Z [main] INFO c.datastax.driver.core.ClockFactory RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null Using java.lang.System clock to generate timestamps. 2021-08-11T16:02:42.650Z [main] INFO o.o.s.b.d.cassandra.CassandraClient RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ** CassandraClient created 2021-08-11T16:02:42.907Z [main] INFO c.d.d.c.p.DCAwareRoundRobinPolicy RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null Using provided data-center name 'Pod' for DCAwareRoundRobinPolicy 2021-08-11T16:02:42.908Z [main] INFO com.datastax.driver.core.Cluster RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null New Cassandra host cassandra.onap/10.242.110.137:9042 added 2021-08-11T16:02:42.908Z [main] INFO com.datastax.driver.core.Cluster RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null New Cassandra host cassandra.onap/10.242.154.72:9042 added 2021-08-11T16:02:42.908Z [main] INFO com.datastax.driver.core.Cluster RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null New Cassandra host cassandra.onap/10.242.184.131:9042 added 2021-08-11T16:02:43.609Z [main] INFO o.o.s.b.d.c.AuditCassandraDao RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ** AuditCassandraDao created 2021-08-11T16:02:43.799Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=80dd9a57-8968-4e71-a8cd-6f55a52fc3b2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:02:43.799Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=80dd9a57-8968-4e71-a8cd-6f55a52fc3b2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@36be114d 2021-08-11T16:02:43.799Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=80dd9a57-8968-4e71-a8cd-6f55a52fc3b2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:02:43.800Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=80dd9a57-8968-4e71-a8cd-6f55a52fc3b2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:02:43.768Z [main] INFO o.o.s.b.c.d.e.DmaapClientFactory RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:02:43.859Z [main] ERROR o.o.s.be.catalog.impl.DmaapProducer RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:02:45.301Z [main] INFO o.o.s.b.d.c.ArtifactCassandraDao RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ** ArtifactCassandraDao created 2021-08-11T16:02:45.827Z [main] INFO o.o.s.b.d.c.SdcSchemaFilesCassandraDao RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ** SdcSchemaFilesCassandraDao created 2021-08-11T16:02:46.071Z [main] WARN o.o.s.b.c.s.CertificateManagerImpl RequestID=71f0ff72-16bc-45da-b6f0-46f773db925f ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=SDC catalog TargetServiceName=null ErrorCode=500 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null Environment variable 'SDC_CERT_DIR' was not provided. Could not load certificates. 2021-08-11T16:02:46.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8315989-a9d2-4946-8928-4f0068ce54db PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:02:46.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8315989-a9d2-4946-8928-4f0068ce54db PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7ca17e21 2021-08-11T16:02:46.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8315989-a9d2-4946-8928-4f0068ce54db PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:02:46.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8315989-a9d2-4946-8928-4f0068ce54db PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:02:48.263Z [main] INFO o.o.s.b.c.d.e.EnvironmentsEngine RequestID=157375e9-7ab3-48e0-baca-7969b402c4ac PartnerName=SDC.BE Environments engine is disabled 2021-08-11T16:02:48.326Z [pool-17-thread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/topics (anonymous) ... 2021-08-11T16:02:48.337Z [pool-17-thread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:02:48.369Z [pool-17-thread-1] WARN c.att.nsa.apiClient.http.HttpClient POST http://message-router.onap:3904/topics/create will send credentials over a clear channel. 2021-08-11T16:02:48.369Z [pool-17-thread-1] INFO c.att.nsa.apiClient.http.HttpClient POST http://message-router.onap:3904/topics/create (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:02:48.485Z [pool-17-thread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:02:48.911Z [pool-17-thread-1] INFO com.datastax.driver.core.utils.UUIDs PID obtained through native call to getpid(): 116 2021-08-11T16:02:48.978Z [pool-17-thread-1] WARN c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "CreateDistributionTopic" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-NOTIF-TOPIC-AUTO" STATUS = "CREATED" PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/producers/iPIxkpAMI8qTcQj8 will send credentials over a clear channel. 2021-08-11T16:02:48.978Z [pool-17-thread-1] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "CreateDistributionTopic" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-NOTIF-TOPIC-AUTO" STATUS = "CREATED" PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/producers/iPIxkpAMI8qTcQj8 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:02:49.005Z [pool-17-thread-1] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "CreateDistributionTopic" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-NOTIF-TOPIC-AUTO" STATUS = "CREATED" \t--> HTTP/1.1 200 OK 2021-08-11T16:02:49.021Z [pool-17-thread-1] WARN c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-NOTIF-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "iPIxkpAMI8qTcQj8" STATUS = "200" POST http://message-router.onap:3904/topics/create will send credentials over a clear channel. 2021-08-11T16:02:49.021Z [pool-17-thread-1] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-NOTIF-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "iPIxkpAMI8qTcQj8" STATUS = "200" POST http://message-router.onap:3904/topics/create (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:02:49.226Z [pool-17-thread-1] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-NOTIF-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "iPIxkpAMI8qTcQj8" STATUS = "200" \t--> HTTP/1.1 200 OK 2021-08-11T16:02:49.248Z [pool-17-thread-1] WARN c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "CreateDistributionTopic" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" STATUS = "CREATED" PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/consumers/iPIxkpAMI8qTcQj8 will send credentials over a clear channel. 2021-08-11T16:02:49.248Z [pool-17-thread-1] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "CreateDistributionTopic" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" STATUS = "CREATED" PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/consumers/iPIxkpAMI8qTcQj8 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:02:49.280Z [pool-17-thread-1] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "CreateDistributionTopic" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" STATUS = "CREATED" \t--> HTTP/1.1 200 OK 2021-08-11T16:02:49.290Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:02:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:02:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:02:50.253Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=26f53eb6-1eef-4703-a05d-9db7c91a85b5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:02:50.253Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=26f53eb6-1eef-4703-a05d-9db7c91a85b5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@675c6de2 2021-08-11T16:02:50.253Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=26f53eb6-1eef-4703-a05d-9db7c91a85b5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:02:50.253Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=26f53eb6-1eef-4703-a05d-9db7c91a85b5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:02:52.369Z [main] INFO o.o.s.b.c.s.AsdcComponentsCleanerTask RequestID=157375e9-7ab3-48e0-baca-7969b402c4ac PartnerName=SDC.BE ServerIPAddress=10.242.24.223 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Enter init method of AsdcComponentsCleaner 2021-08-11T16:02:52.371Z [main] INFO o.o.s.b.c.s.AsdcComponentsCleanerTask RequestID=157375e9-7ab3-48e0-baca-7969b402c4ac PartnerName=SDC.BE ServerIPAddress=10.242.24.223 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p End init method of AsdcComponentsCleaner 2021-08-11T16:02:52.426Z [main] INFO o.s.b.f.a.AutowiredAnnotationBeanPostProcessor RequestID=157375e9-7ab3-48e0-baca-7969b402c4ac PartnerName=SDC.BE ServerIPAddress=10.242.24.223 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Autowired annotation is not supported on static methods: public static void org.openecomp.sdc.be.components.impl.ImportUtils.setComponentsUtils(org.openecomp.sdc.be.impl.ComponentsUtils) 2021-08-11T16:02:53.568Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf8d2d94-4478-4a47-8601-86d197a09749 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:02:53.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf8d2d94-4478-4a47-8601-86d197a09749 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1082102b 2021-08-11T16:02:53.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf8d2d94-4478-4a47-8601-86d197a09749 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:02:53.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf8d2d94-4478-4a47-8601-86d197a09749 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:02:53.574Z [main] INFO o.o.s.b.d.cassandra.FeatureToggleDao RequestID=157375e9-7ab3-48e0-baca-7969b402c4ac PartnerName=SDC.BE ServerIPAddress=10.242.24.223 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ** FeatureToggleDao created 2021-08-11T16:02:54.165Z [main] INFO o.s.web.context.ContextLoader RequestID=157375e9-7ab3-48e0-baca-7969b402c4ac PartnerName=SDC.BE ServerIPAddress=10.242.24.223 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Root WebApplicationContext initialized in 24325 ms 2021-08-11T16:02:56.571Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7f10362-7006-4f53-880e-9046a2c66389 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:02:56.571Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7f10362-7006-4f53-880e-9046a2c66389 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1e9306cc 2021-08-11T16:02:56.572Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7f10362-7006-4f53-880e-9046a2c66389 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:02:56.572Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7f10362-7006-4f53-880e-9046a2c66389 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:02:57.266Z [main] INFO o.a.c.b.FluentPropertyBeanIntrospector RequestID=157375e9-7ab3-48e0-baca-7969b402c4ac PartnerName=SDC.BE ServerIPAddress=10.242.24.223 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Error when creating PropertyDescriptor for public final void org.apache.commons.configuration2.AbstractConfiguration.setProperty(java.lang.String,java.lang.Object)! Ignoring this property. 2021-08-11T16:02:57.283Z [main] INFO o.a.c.b.FluentPropertyBeanIntrospector RequestID=157375e9-7ab3-48e0-baca-7969b402c4ac PartnerName=SDC.BE ServerIPAddress=10.242.24.223 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Error when creating PropertyDescriptor for public final void org.apache.commons.configuration2.AbstractConfiguration.setProperty(java.lang.String,java.lang.Object)! Ignoring this property. 2021-08-11T16:02:57.310Z [main] INFO o.a.c.b.FluentPropertyBeanIntrospector RequestID=157375e9-7ab3-48e0-baca-7969b402c4ac PartnerName=SDC.BE ServerIPAddress=10.242.24.223 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Error when creating PropertyDescriptor for public final void org.apache.commons.configuration2.AbstractConfiguration.setProperty(java.lang.String,java.lang.Object)! Ignoring this property. 2021-08-11T16:02:59.149Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=46858472-401e-4db8-bb41-2e1be8b45556 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:02:59.149Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=46858472-401e-4db8-bb41-2e1be8b45556 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@28adf0e9 2021-08-11T16:02:59.149Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=46858472-401e-4db8-bb41-2e1be8b45556 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:02:59.149Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=46858472-401e-4db8-bb41-2e1be8b45556 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:03:00.141Z [main] INFO o.h.validator.internal.util.Version RequestID=157375e9-7ab3-48e0-baca-7969b402c4ac PartnerName=SDC.BE ServerIPAddress=10.242.24.223 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p HV000001: Hibernate Validator 5.3.6.Final 2021-08-11T16:03:00.976Z [qtp2045766957-16] INFO com.jcabi.aspects.aj.NamedThreads RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/healthCheck EntryTimestamp=2021-08-11T16:03:00.829Z InvocationID=f18f8f9d-0e28-4965-97f7-c1fe6e6a6735 PartnerName=kube-probe/1.20 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p jcabi-aspects 0.20.1/f0aff93 started new daemon thread jcabi-loggable for watching of @Loggable annotated methods 2021-08-11T16:03:01.691Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5348d7ca-0201-4626-8c9d-9d0a8b022e78 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:03:01.692Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5348d7ca-0201-4626-8c9d-9d0a8b022e78 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@59757377 2021-08-11T16:03:01.692Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5348d7ca-0201-4626-8c9d-9d0a8b022e78 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:03:01.692Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5348d7ca-0201-4626-8c9d-9d0a8b022e78 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:03:03.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:03:03.331Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:03:03.332Z [UEB-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL ErrorCategory=INFO RequestID=null ServiceName=uebHealthCheck TargetServiceName=null ErrorCode=200 TargetEntity=uebHealthCheck ASDC Backend connection recovery to UEB Cluster. 2021-08-11T16:03:04.427Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:03:04.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=569cb4d3-64ec-4cb4-acab-16f414b63f7b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:03:04.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=569cb4d3-64ec-4cb4-acab-16f414b63f7b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7d051fbc 2021-08-11T16:03:04.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=569cb4d3-64ec-4cb4-acab-16f414b63f7b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:03:04.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=569cb4d3-64ec-4cb4-acab-16f414b63f7b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:03:04.782Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=569cb4d3-64ec-4cb4-acab-16f414b63f7b ErrorCategory=INFO ServiceName=BE Component DE state changed from UNKNOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component DE state changed from UNKNOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:03:04.783Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=569cb4d3-64ec-4cb4-acab-16f414b63f7b ErrorCategory=INFO ServiceName=BE Component DE state changed from UNKNOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:03:07.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0b6218cc-8cfb-4a0a-aea7-6a80987ec870 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:03:07.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0b6218cc-8cfb-4a0a-aea7-6a80987ec870 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7ba6c3b6 2021-08-11T16:03:07.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0b6218cc-8cfb-4a0a-aea7-6a80987ec870 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:03:07.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0b6218cc-8cfb-4a0a-aea7-6a80987ec870 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:03:10.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b13294cc-711f-41ea-a830-57a5f89949d8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:03:10.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b13294cc-711f-41ea-a830-57a5f89949d8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3ccb4e93 2021-08-11T16:03:10.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b13294cc-711f-41ea-a830-57a5f89949d8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:03:10.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b13294cc-711f-41ea-a830-57a5f89949d8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:03:13.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fccc64e3-a03c-41cd-a2cc-ec12ea562a89 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:03:13.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fccc64e3-a03c-41cd-a2cc-ec12ea562a89 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4e577cb6 2021-08-11T16:03:13.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fccc64e3-a03c-41cd-a2cc-ec12ea562a89 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:03:13.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fccc64e3-a03c-41cd-a2cc-ec12ea562a89 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:03:16.688Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7095fdee-5414-4983-b1fc-0c958197e2a1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:03:16.689Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7095fdee-5414-4983-b1fc-0c958197e2a1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1b7a623f 2021-08-11T16:03:16.689Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7095fdee-5414-4983-b1fc-0c958197e2a1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:03:16.689Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7095fdee-5414-4983-b1fc-0c958197e2a1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:03:18.310Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:03:18.318Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:03:19.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6852c4fa-c2e6-45aa-bc60-df6e0f8be9d9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:03:19.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6852c4fa-c2e6-45aa-bc60-df6e0f8be9d9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@560ca995 2021-08-11T16:03:19.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6852c4fa-c2e6-45aa-bc60-df6e0f8be9d9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:03:19.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6852c4fa-c2e6-45aa-bc60-df6e0f8be9d9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:03:22.965Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d836e5b9-7eae-4b71-9cac-3df1c9804f93 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:03:22.966Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d836e5b9-7eae-4b71-9cac-3df1c9804f93 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3055da11 2021-08-11T16:03:22.966Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d836e5b9-7eae-4b71-9cac-3df1c9804f93 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:03:22.967Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d836e5b9-7eae-4b71-9cac-3df1c9804f93 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:03:25.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1bc46714-a032-4df8-9233-8798646349c5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:03:25.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1bc46714-a032-4df8-9233-8798646349c5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4592a261 2021-08-11T16:03:25.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1bc46714-a032-4df8-9233-8798646349c5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:03:25.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1bc46714-a032-4df8-9233-8798646349c5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:03:28.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f383aa9a-b1fb-40ec-ae35-eb91a3fb482d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:03:28.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f383aa9a-b1fb-40ec-ae35-eb91a3fb482d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@d315b76 2021-08-11T16:03:28.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f383aa9a-b1fb-40ec-ae35-eb91a3fb482d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:03:28.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f383aa9a-b1fb-40ec-ae35-eb91a3fb482d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:03:31.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=130fe8b0-b19e-44f5-bb38-9ea0ac9a4237 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:03:31.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=130fe8b0-b19e-44f5-bb38-9ea0ac9a4237 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3ec20b44 2021-08-11T16:03:31.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=130fe8b0-b19e-44f5-bb38-9ea0ac9a4237 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:03:31.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=130fe8b0-b19e-44f5-bb38-9ea0ac9a4237 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:03:33.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:03:33.331Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:03:34.628Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f99c98c-e5f3-4190-bcfe-4e84e2ad4928 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:03:34.628Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f99c98c-e5f3-4190-bcfe-4e84e2ad4928 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@20d9dbdf 2021-08-11T16:03:34.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f99c98c-e5f3-4190-bcfe-4e84e2ad4928 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:03:34.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f99c98c-e5f3-4190-bcfe-4e84e2ad4928 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:03:37.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0bc356f-054f-43e5-8651-68b449efc7e7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:03:37.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0bc356f-054f-43e5-8651-68b449efc7e7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7088e40f 2021-08-11T16:03:37.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0bc356f-054f-43e5-8651-68b449efc7e7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:03:37.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0bc356f-054f-43e5-8651-68b449efc7e7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:03:40.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4fc31703-758d-4e03-bab3-b6cee9ab80a5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:03:40.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4fc31703-758d-4e03-bab3-b6cee9ab80a5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7765eb83 2021-08-11T16:03:40.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4fc31703-758d-4e03-bab3-b6cee9ab80a5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:03:40.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4fc31703-758d-4e03-bab3-b6cee9ab80a5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:03:43.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b86410b-fd1d-472c-af7a-2f2c90d07e03 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:03:43.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b86410b-fd1d-472c-af7a-2f2c90d07e03 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@59d35600 2021-08-11T16:03:43.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b86410b-fd1d-472c-af7a-2f2c90d07e03 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:03:43.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b86410b-fd1d-472c-af7a-2f2c90d07e03 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:03:46.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a9177e5-b8f8-4d8e-a010-94f2a814f863 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:03:46.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a9177e5-b8f8-4d8e-a010-94f2a814f863 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@37ba79c6 2021-08-11T16:03:46.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a9177e5-b8f8-4d8e-a010-94f2a814f863 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:03:46.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a9177e5-b8f8-4d8e-a010-94f2a814f863 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:03:48.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:03:48.318Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:03:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:03:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:03:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:03:50.053Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=de6e9f30-5cdd-4aa3-99fd-64b55f58a1b0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:03:50.053Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=de6e9f30-5cdd-4aa3-99fd-64b55f58a1b0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@254ba2 2021-08-11T16:03:50.053Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=de6e9f30-5cdd-4aa3-99fd-64b55f58a1b0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:03:50.053Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=de6e9f30-5cdd-4aa3-99fd-64b55f58a1b0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:03:53.074Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7e0d663b-403d-4818-84eb-a9b4c1784b43 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:03:53.074Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7e0d663b-403d-4818-84eb-a9b4c1784b43 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@584481d6 2021-08-11T16:03:53.075Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7e0d663b-403d-4818-84eb-a9b4c1784b43 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:03:53.075Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7e0d663b-403d-4818-84eb-a9b4c1784b43 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:03:55.805Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=954dafa8-7ff8-4347-9d00-4638ad07eb21 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:03:55.805Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=954dafa8-7ff8-4347-9d00-4638ad07eb21 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1be4bf88 2021-08-11T16:03:55.805Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=954dafa8-7ff8-4347-9d00-4638ad07eb21 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:03:55.805Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=954dafa8-7ff8-4347-9d00-4638ad07eb21 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:03:58.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a93a3796-a819-4d4d-8a65-070c4113715a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:03:58.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a93a3796-a819-4d4d-8a65-070c4113715a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7f74917b 2021-08-11T16:03:58.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a93a3796-a819-4d4d-8a65-070c4113715a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:03:58.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a93a3796-a819-4d4d-8a65-070c4113715a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:04:01.639Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2eab8a3-fadc-4967-956a-b2a97752fff1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:04:01.639Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2eab8a3-fadc-4967-956a-b2a97752fff1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5526bd6c 2021-08-11T16:04:01.639Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2eab8a3-fadc-4967-956a-b2a97752fff1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:04:01.639Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2eab8a3-fadc-4967-956a-b2a97752fff1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:04:03.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:04:03.321Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:04:04.306Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:04:04.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=861aa446-a3a2-43f5-9f8c-da0bb80495b0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:04:04.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=861aa446-a3a2-43f5-9f8c-da0bb80495b0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2910cc2f 2021-08-11T16:04:04.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=861aa446-a3a2-43f5-9f8c-da0bb80495b0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:04:04.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=861aa446-a3a2-43f5-9f8c-da0bb80495b0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:04:07.636Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7e800d5-c21b-4dbe-94b9-e40e4fea9830 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:04:07.636Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7e800d5-c21b-4dbe-94b9-e40e4fea9830 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@17b62e7a 2021-08-11T16:04:07.636Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7e800d5-c21b-4dbe-94b9-e40e4fea9830 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:04:07.637Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7e800d5-c21b-4dbe-94b9-e40e4fea9830 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:04:10.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0389c35e-8c89-4f9a-8621-f0e2ce18fa83 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:04:10.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0389c35e-8c89-4f9a-8621-f0e2ce18fa83 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4c928abe 2021-08-11T16:04:10.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0389c35e-8c89-4f9a-8621-f0e2ce18fa83 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:04:10.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0389c35e-8c89-4f9a-8621-f0e2ce18fa83 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:04:13.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f6267c7-6e99-425f-9adc-66330695eae3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:04:13.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f6267c7-6e99-425f-9adc-66330695eae3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2c3f939d 2021-08-11T16:04:13.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f6267c7-6e99-425f-9adc-66330695eae3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:04:13.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f6267c7-6e99-425f-9adc-66330695eae3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:04:16.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d604616b-4ee5-4464-8c66-b4d69073a49b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:04:16.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d604616b-4ee5-4464-8c66-b4d69073a49b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3e15381 2021-08-11T16:04:16.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d604616b-4ee5-4464-8c66-b4d69073a49b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:04:16.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d604616b-4ee5-4464-8c66-b4d69073a49b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:04:18.309Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:04:18.325Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:04:19.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f949f4c6-fd4e-4b96-9cc7-dbc8065c2391 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:04:19.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f949f4c6-fd4e-4b96-9cc7-dbc8065c2391 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@25027879 2021-08-11T16:04:19.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f949f4c6-fd4e-4b96-9cc7-dbc8065c2391 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:04:19.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f949f4c6-fd4e-4b96-9cc7-dbc8065c2391 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:04:22.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6942013d-b9f7-4af0-8b3a-ee74c809cec8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:04:22.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6942013d-b9f7-4af0-8b3a-ee74c809cec8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7d1dd11d 2021-08-11T16:04:22.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6942013d-b9f7-4af0-8b3a-ee74c809cec8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:04:22.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6942013d-b9f7-4af0-8b3a-ee74c809cec8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:04:25.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b620a6ff-5831-405c-9b26-773e12bb76cf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:04:25.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b620a6ff-5831-405c-9b26-773e12bb76cf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2d6e0658 2021-08-11T16:04:25.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b620a6ff-5831-405c-9b26-773e12bb76cf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:04:25.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b620a6ff-5831-405c-9b26-773e12bb76cf PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:04:28.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a00c259a-7c73-4084-a9b4-b2fd361e5b3f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:04:28.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a00c259a-7c73-4084-a9b4-b2fd361e5b3f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@249a352f 2021-08-11T16:04:28.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a00c259a-7c73-4084-a9b4-b2fd361e5b3f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:04:28.959Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a00c259a-7c73-4084-a9b4-b2fd361e5b3f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:04:31.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=820f8648-3b52-4aaa-9498-b6a2246312e7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:04:31.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=820f8648-3b52-4aaa-9498-b6a2246312e7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@12ed4075 2021-08-11T16:04:31.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=820f8648-3b52-4aaa-9498-b6a2246312e7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:04:31.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=820f8648-3b52-4aaa-9498-b6a2246312e7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:04:33.309Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:04:33.336Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:04:34.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8aecea03-4ac7-45ac-9c23-732b76109b83 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:04:34.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8aecea03-4ac7-45ac-9c23-732b76109b83 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1ebfe0c 2021-08-11T16:04:34.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8aecea03-4ac7-45ac-9c23-732b76109b83 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:04:34.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8aecea03-4ac7-45ac-9c23-732b76109b83 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:04:37.635Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56726799-828a-4eab-bd76-a3434d8e2461 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:04:37.635Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56726799-828a-4eab-bd76-a3434d8e2461 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5c6a64af 2021-08-11T16:04:37.635Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56726799-828a-4eab-bd76-a3434d8e2461 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:04:37.636Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56726799-828a-4eab-bd76-a3434d8e2461 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:04:40.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f605ba16-0e65-4762-9a08-38881b0b4f7b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:04:40.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f605ba16-0e65-4762-9a08-38881b0b4f7b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@29ac0140 2021-08-11T16:04:40.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f605ba16-0e65-4762-9a08-38881b0b4f7b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:04:40.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f605ba16-0e65-4762-9a08-38881b0b4f7b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:04:43.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd67ff13-df4b-4c8b-84ac-2b5ede7181a4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:04:43.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd67ff13-df4b-4c8b-84ac-2b5ede7181a4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@51ba02e8 2021-08-11T16:04:43.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd67ff13-df4b-4c8b-84ac-2b5ede7181a4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:04:43.639Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd67ff13-df4b-4c8b-84ac-2b5ede7181a4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:04:46.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f7a9564-87f3-4719-b38c-201f45af3067 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:04:46.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f7a9564-87f3-4719-b38c-201f45af3067 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@fff9d4d 2021-08-11T16:04:46.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f7a9564-87f3-4719-b38c-201f45af3067 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:04:46.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f7a9564-87f3-4719-b38c-201f45af3067 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:04:48.316Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:04:48.367Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:04:49.291Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:04:49.292Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:04:49.292Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:04:50.151Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dde56a61-e658-45dd-b80f-5ff07c0fe548 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:04:50.151Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dde56a61-e658-45dd-b80f-5ff07c0fe548 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3fdf01 2021-08-11T16:04:50.151Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dde56a61-e658-45dd-b80f-5ff07c0fe548 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:04:50.151Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dde56a61-e658-45dd-b80f-5ff07c0fe548 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:04:53.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b035fbc2-7c4d-4da9-b311-64f03c8a9d4c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:04:53.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b035fbc2-7c4d-4da9-b311-64f03c8a9d4c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@49d07847 2021-08-11T16:04:53.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b035fbc2-7c4d-4da9-b311-64f03c8a9d4c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:04:53.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b035fbc2-7c4d-4da9-b311-64f03c8a9d4c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:04:56.962Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31d8ca7f-1c4c-4c35-95f2-1fb6bd834869 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:04:56.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31d8ca7f-1c4c-4c35-95f2-1fb6bd834869 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@487c9ed7 2021-08-11T16:04:56.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31d8ca7f-1c4c-4c35-95f2-1fb6bd834869 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:04:56.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31d8ca7f-1c4c-4c35-95f2-1fb6bd834869 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:04:59.279Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e852222c-0537-4ef7-8d42-799b61f8a81c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:04:59.279Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e852222c-0537-4ef7-8d42-799b61f8a81c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3b1a7760 2021-08-11T16:04:59.279Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e852222c-0537-4ef7-8d42-799b61f8a81c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:04:59.280Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e852222c-0537-4ef7-8d42-799b61f8a81c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:05:01.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=61827447-8335-4d92-bd7a-e75ba13f052c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:05:01.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=61827447-8335-4d92-bd7a-e75ba13f052c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@452360d5 2021-08-11T16:05:01.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=61827447-8335-4d92-bd7a-e75ba13f052c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:05:01.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=61827447-8335-4d92-bd7a-e75ba13f052c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:05:03.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:05:03.319Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:05:04.346Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:05:04.652Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ab424f1-1dfb-4dcd-82a9-872f5a049797 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:05:04.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ab424f1-1dfb-4dcd-82a9-872f5a049797 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@ffb2a76 2021-08-11T16:05:04.654Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ab424f1-1dfb-4dcd-82a9-872f5a049797 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:05:04.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ab424f1-1dfb-4dcd-82a9-872f5a049797 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:05:07.604Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=751e83ae-1887-49a3-8bc7-e63a64e361fa PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:05:07.604Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=751e83ae-1887-49a3-8bc7-e63a64e361fa PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@23dd8778 2021-08-11T16:05:07.604Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=751e83ae-1887-49a3-8bc7-e63a64e361fa PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:05:07.604Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=751e83ae-1887-49a3-8bc7-e63a64e361fa PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:05:10.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe1cc72b-aaa2-4d0a-aa95-8291dd9cf20c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:05:10.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe1cc72b-aaa2-4d0a-aa95-8291dd9cf20c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4b944514 2021-08-11T16:05:10.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe1cc72b-aaa2-4d0a-aa95-8291dd9cf20c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:05:10.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe1cc72b-aaa2-4d0a-aa95-8291dd9cf20c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:05:13.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e132072-1e14-45fc-b16e-ef9db3c2768c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:05:13.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e132072-1e14-45fc-b16e-ef9db3c2768c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@156fa17b 2021-08-11T16:05:13.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e132072-1e14-45fc-b16e-ef9db3c2768c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:05:13.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e132072-1e14-45fc-b16e-ef9db3c2768c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:05:16.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b04a939-fd09-4fe6-bf11-b3e10fd52630 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:05:16.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b04a939-fd09-4fe6-bf11-b3e10fd52630 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4311813a 2021-08-11T16:05:16.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b04a939-fd09-4fe6-bf11-b3e10fd52630 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:05:16.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b04a939-fd09-4fe6-bf11-b3e10fd52630 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:05:18.310Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:05:18.318Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:05:19.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c1645b2-0547-425e-bb88-f9a8e41cede6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:05:19.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c1645b2-0547-425e-bb88-f9a8e41cede6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@cb82867 2021-08-11T16:05:19.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c1645b2-0547-425e-bb88-f9a8e41cede6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:05:19.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c1645b2-0547-425e-bb88-f9a8e41cede6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:05:21.554Z [qtp2045766957-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=47b48324-efb4-4966-b5c9-135db991703a InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=2 Severity=0 PartnerName=multicloud ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.23.16 StatusCode=INPROGRESS RemoteHost=10.242.23.16 ServiceName=v1/artifactTypes ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:21.545842Z InvocationID=236625e7-a9ef-4c91-b180-ccd68b5d9330 LogTimestamp=2021-08-11T16:05:21.548392Z Entering 2021-08-11T16:05:21.555Z [qtp2045766957-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=47b48324-efb4-4966-b5c9-135db991703a InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=2 Severity=0 PartnerName=multicloud ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.23.16 StatusCode=INPROGRESS userId=null RemoteHost=10.242.23.16 ServiceName=v1/artifactTypes ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:21.545842Z InvocationID=236625e7-a9ef-4c91-b180-ccd68b5d9330 LogTimestamp=2021-08-11T16:05:21.548392Z localAddr=10.242.24.223 remoteAddr=10.242.23.16 GET /sdc/v1/artifactTypes HTTP/1.1 2021-08-11T16:05:21.886Z [qtp2045766957-20] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=329 PartnerName=Apache-HttpClient/4.5.7 (Java/11.0.8) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/artifactTypes EntryTimestamp=2021-08-11T16:05:21.556Z InvocationID=c3e4b613-66e8-4869-bae2-bfb16b2d22e8 LogTimestamp=2021-08-11T16:05:21.885645Z ResponseDesc=OK Exiting. 2021-08-11T16:05:21.887Z [qtp2045766957-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc/v1/artifactTypes HTTP/1.1 SC="200" 2021-08-11T16:05:22.128Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=da62dc24-514e-4aea-8d67-129d8ca03120 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=multicloud auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.23.16 StatusCode=INPROGRESS RemoteHost=10.242.23.16 ServerIPAddress=10.242.24.223 ServiceName=v1/registerForDistribution ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:22.127701Z InvocationID=b59e25e2-0f47-4240-8db8-2f741a77ea7e LogTimestamp=2021-08-11T16:05:22.127792Z Entering 2021-08-11T16:05:22.128Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=da62dc24-514e-4aea-8d67-129d8ca03120 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=multicloud auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.23.16 StatusCode=INPROGRESS userId=null RemoteHost=10.242.23.16 ServerIPAddress=10.242.24.223 ServiceName=v1/registerForDistribution ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:22.127701Z InvocationID=b59e25e2-0f47-4240-8db8-2f741a77ea7e LogTimestamp=2021-08-11T16:05:22.127792Z localAddr=10.242.24.223 remoteAddr=10.242.23.16 POST /sdc/v1/registerForDistribution HTTP/1.1 2021-08-11T16:05:22.159Z [qtp2045766957-17] WARN c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:05:22.128Z InvocationID=7f441413-fcf7-49b6-9b44-190602e4040a PartnerName=Apache-HttpClient/4.5.7 (Java/11.0.8) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/producers/zYeLkgFHgDYgfTrm will send credentials over a clear channel. 2021-08-11T16:05:22.159Z [qtp2045766957-17] INFO c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:05:22.128Z InvocationID=7f441413-fcf7-49b6-9b44-190602e4040a PartnerName=Apache-HttpClient/4.5.7 (Java/11.0.8) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/producers/zYeLkgFHgDYgfTrm (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:05:22.234Z [qtp2045766957-17] INFO c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:05:22.128Z InvocationID=7f441413-fcf7-49b6-9b44-190602e4040a PartnerName=Apache-HttpClient/4.5.7 (Java/11.0.8) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p \t--> HTTP/1.1 200 OK 2021-08-11T16:05:22.246Z [qtp2045766957-17] WARN c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "zYeLkgFHgDYgfTrm" STATUS = "200" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:05:22.128Z InvocationID=7f441413-fcf7-49b6-9b44-190602e4040a PartnerName=Apache-HttpClient/4.5.7 (Java/11.0.8) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/consumers/zYeLkgFHgDYgfTrm will send credentials over a clear channel. 2021-08-11T16:05:22.246Z [qtp2045766957-17] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "zYeLkgFHgDYgfTrm" STATUS = "200" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:05:22.128Z InvocationID=7f441413-fcf7-49b6-9b44-190602e4040a PartnerName=Apache-HttpClient/4.5.7 (Java/11.0.8) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/consumers/zYeLkgFHgDYgfTrm (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:05:22.264Z [qtp2045766957-17] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "zYeLkgFHgDYgfTrm" STATUS = "200" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:05:22.128Z InvocationID=7f441413-fcf7-49b6-9b44-190602e4040a PartnerName=Apache-HttpClient/4.5.7 (Java/11.0.8) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p \t--> HTTP/1.1 200 OK 2021-08-11T16:05:22.278Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "DRegister" CONSUMER_ID = "multicloud-k8s-id" API_KEY = "zYeLkgFHgDYgfTrm" D_ENV = "AUTO" STATUS = "200" DESC = "" DNOTIF_TOPIC = "SDC-DISTR-NOTIF-TOPIC-AUTO" DSTATUS_TOPIC = "SDC-DISTR-STATUS-TOPIC-AUTO" ResponseCode=200 RequestID=null ElapsedTime=149 PartnerName=Apache-HttpClient/4.5.7 (Java/11.0.8) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:05:22.128Z InvocationID=7f441413-fcf7-49b6-9b44-190602e4040a LogTimestamp=2021-08-11T16:05:22.277947Z ResponseDesc=OK Exiting. 2021-08-11T16:05:22.278Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc/v1/registerForDistribution HTTP/1.1 SC="200" 2021-08-11T16:05:22.299Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=1d7ce433-fd71-4bc0-993c-48242090b3aa InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/user/jh0003 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:22.298698Z InvocationID=6a3eaddc-70d1-4ec1-a55c-b9e1be9c65a2 LogTimestamp=2021-08-11T16:05:22.298865Z Entering 2021-08-11T16:05:22.299Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=1d7ce433-fd71-4bc0-993c-48242090b3aa InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/user/jh0003 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:22.298698Z InvocationID=6a3eaddc-70d1-4ec1-a55c-b9e1be9c65a2 LogTimestamp=2021-08-11T16:05:22.298865Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-08-11T16:05:22.352Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=52 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/user/jh0003 EntryTimestamp=2021-08-11T16:05:22.299Z InvocationID=55213e64-a92c-4c6a-ad9f-1d74a594fd49 LogTimestamp=2021-08-11T16:05:22.351893Z ResponseDesc=OK Exiting. 2021-08-11T16:05:22.352Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2021-08-11T16:05:22.402Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 RequestID=null ElapsedTime=20854 PartnerName=kube-probe/1.20 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=ERROR ServerIPAddress=10.242.24.223 ServiceName=/healthCheck ErrorDesc=Internal Server Error EntryTimestamp=2021-08-11T16:05:01.547Z InvocationID=58e9db61-fbb3-49fb-98b4-fd2755309335 ErrorCode=500 LogTimestamp=2021-08-11T16:05:22.401690Z ResponseDesc=Internal Server Error Exiting. 2021-08-11T16:05:22.403Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userdemo HTTP/1.1 SC="500" 2021-08-11T16:05:22.417Z [qtp2045766957-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=888dc0cf-5c79-4729-944e-74996b0625f2 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/user ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:22.417051Z InvocationID=715728d6-a62a-411e-9f06-57333f5f5fd3 LogTimestamp=2021-08-11T16:05:22.417133Z Entering 2021-08-11T16:05:22.417Z [qtp2045766957-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=888dc0cf-5c79-4729-944e-74996b0625f2 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/user ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:22.417051Z InvocationID=715728d6-a62a-411e-9f06-57333f5f5fd3 LogTimestamp=2021-08-11T16:05:22.417133Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/user HTTP/1.1 2021-08-11T16:05:22.537Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=c43915ba-ad9d-4ca1-b9d0-151e01db3942 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=policy ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.24.27 StatusCode=INPROGRESS RemoteHost=10.242.24.27 ServiceName=v1/artifactTypes ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:22.537201Z InvocationID=a30ac40f-bd2d-4648-a1f9-d43fd7c45dbe LogTimestamp=2021-08-11T16:05:22.537293Z Entering 2021-08-11T16:05:22.537Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=c43915ba-ad9d-4ca1-b9d0-151e01db3942 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=policy ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.24.27 StatusCode=INPROGRESS userId=null RemoteHost=10.242.24.27 ServiceName=v1/artifactTypes ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:22.537201Z InvocationID=a30ac40f-bd2d-4648-a1f9-d43fd7c45dbe LogTimestamp=2021-08-11T16:05:22.537293Z localAddr=10.242.24.223 remoteAddr=10.242.24.27 GET /sdc/v1/artifactTypes HTTP/1.1 2021-08-11T16:05:22.539Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=1 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/artifactTypes EntryTimestamp=2021-08-11T16:05:22.538Z InvocationID=19ab1b30-45b0-4dd4-b4ec-d7bb9a1356cb LogTimestamp=2021-08-11T16:05:22.539500Z ResponseDesc=OK Exiting. 2021-08-11T16:05:22.540Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc/v1/artifactTypes HTTP/1.1 SC="200" 2021-08-11T16:05:22.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=211d48b3-f928-4ab9-8949-6c08e737a6b1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:05:22.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=211d48b3-f928-4ab9-8949-6c08e737a6b1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@65ed7001 2021-08-11T16:05:22.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=211d48b3-f928-4ab9-8949-6c08e737a6b1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:05:22.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=211d48b3-f928-4ab9-8949-6c08e737a6b1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:05:22.831Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=30f987b4-76e1-4d4a-b558-d628aa7d3787 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=policy ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.24.27 StatusCode=INPROGRESS RemoteHost=10.242.24.27 ServiceName=v1/registerForDistribution ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:22.830348Z InvocationID=dacb05cd-ceda-4d98-a8dd-4740eebb733d LogTimestamp=2021-08-11T16:05:22.830463Z Entering 2021-08-11T16:05:22.831Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=30f987b4-76e1-4d4a-b558-d628aa7d3787 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=policy ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.24.27 StatusCode=INPROGRESS userId=null RemoteHost=10.242.24.27 ServiceName=v1/registerForDistribution ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:22.830348Z InvocationID=dacb05cd-ceda-4d98-a8dd-4740eebb733d LogTimestamp=2021-08-11T16:05:22.830463Z localAddr=10.242.24.223 remoteAddr=10.242.24.27 POST /sdc/v1/registerForDistribution HTTP/1.1 2021-08-11T16:05:22.835Z [qtp2045766957-14] WARN c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:05:22.831Z InvocationID=7ca6d568-59c2-4b32-8419-8931c8466db7 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/producers/Iwv5q5kZKgMqCc7H will send credentials over a clear channel. 2021-08-11T16:05:22.835Z [qtp2045766957-14] INFO c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:05:22.831Z InvocationID=7ca6d568-59c2-4b32-8419-8931c8466db7 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/producers/Iwv5q5kZKgMqCc7H (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:05:22.884Z [qtp2045766957-14] INFO c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:05:22.831Z InvocationID=7ca6d568-59c2-4b32-8419-8931c8466db7 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p \t--> HTTP/1.1 200 OK 2021-08-11T16:05:22.893Z [qtp2045766957-14] WARN c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "Iwv5q5kZKgMqCc7H" STATUS = "200" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:05:22.831Z InvocationID=7ca6d568-59c2-4b32-8419-8931c8466db7 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/consumers/Iwv5q5kZKgMqCc7H will send credentials over a clear channel. 2021-08-11T16:05:22.893Z [qtp2045766957-14] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "Iwv5q5kZKgMqCc7H" STATUS = "200" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:05:22.831Z InvocationID=7ca6d568-59c2-4b32-8419-8931c8466db7 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/consumers/Iwv5q5kZKgMqCc7H (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:05:22.912Z [qtp2045766957-14] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "Iwv5q5kZKgMqCc7H" STATUS = "200" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:05:22.831Z InvocationID=7ca6d568-59c2-4b32-8419-8931c8466db7 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p \t--> HTTP/1.1 200 OK 2021-08-11T16:05:22.918Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "DRegister" CONSUMER_ID = "policy-id" API_KEY = "Iwv5q5kZKgMqCc7H" D_ENV = "AUTO" STATUS = "200" DESC = "" DNOTIF_TOPIC = "SDC-DISTR-NOTIF-TOPIC-AUTO" DSTATUS_TOPIC = "SDC-DISTR-STATUS-TOPIC-AUTO" ResponseCode=200 RequestID=null ElapsedTime=86 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:05:22.831Z InvocationID=7ca6d568-59c2-4b32-8419-8931c8466db7 LogTimestamp=2021-08-11T16:05:22.917751Z ResponseDesc=OK Exiting. 2021-08-11T16:05:22.918Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc/v1/registerForDistribution HTTP/1.1 SC="200" 2021-08-11T16:05:23.774Z [qtp2045766957-20] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "demo demo demo demo@openecomp.org ADMIN" STATUS = "201" DESC = "OK" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/user EntryTimestamp=2021-08-11T16:05:22.417Z InvocationID=eba5ac71-b4b1-4949-b31d-8851e3252803 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:05:23.774Z [qtp2045766957-20] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "demo demo demo demo@openecomp.org ADMIN" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/user TargetServiceName=null EntryTimestamp=2021-08-11T16:05:22.417Z InvocationID=eba5ac71-b4b1-4949-b31d-8851e3252803 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:05:23.776Z [qtp2045766957-20] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "demo demo demo demo@openecomp.org ADMIN" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=1358 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/user TargetServiceName=null EntryTimestamp=2021-08-11T16:05:22.417Z InvocationID=eba5ac71-b4b1-4949-b31d-8851e3252803 ErrorCode=500 LogTimestamp=2021-08-11T16:05:23.775823Z ResponseDesc=Created Exiting. 2021-08-11T16:05:23.776Z [qtp2045766957-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="201" 2021-08-11T16:05:23.807Z [qtp2045766957-11] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:23.806789Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-08-11T16:05:23.807Z [qtp2045766957-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:23.806789Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-08-11T16:05:23.808Z [qtp2045766957-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userop0001 HTTP/1.1 SC="500" 2021-08-11T16:05:23.833Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=9050b74f-099d-458b-ba48-da83d8072042 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/user ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:23.833459Z InvocationID=20122d2f-e63e-4463-b371-a16ad2d1e253 LogTimestamp=2021-08-11T16:05:23.833579Z Entering 2021-08-11T16:05:23.834Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=9050b74f-099d-458b-ba48-da83d8072042 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/user ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:23.833459Z InvocationID=20122d2f-e63e-4463-b371-a16ad2d1e253 LogTimestamp=2021-08-11T16:05:23.833579Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/user HTTP/1.1 2021-08-11T16:05:23.882Z [qtp2045766957-12] ERROR o.o.sdc.be.servlets.UserAdminServlet RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/user EntryTimestamp=2021-08-11T16:05:23.834Z InvocationID=db0bd4a5-690b-4183-bd8b-83fb8203e3a3 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@30ebe', 'User [firstName=Oper, lastName=P, userId=op0001, email=op0001@openecomp.org, role=OPS, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByActionStatusComponentException out of org.openecomp.sdc.be.user.UserBusinessLogic#validateRole[157] in 30ms 2021-08-11T16:05:23.887Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=400 RequestID=null ElapsedTime=52 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/user ErrorDesc=Bad Request EntryTimestamp=2021-08-11T16:05:23.834Z InvocationID=db0bd4a5-690b-4183-bd8b-83fb8203e3a3 ErrorCode=400 LogTimestamp=2021-08-11T16:05:23.886511Z ResponseDesc=Bad Request Exiting. 2021-08-11T16:05:23.887Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="400" 2021-08-11T16:05:23.900Z [qtp2045766957-17] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:23.900794Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-08-11T16:05:23.900Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:23.900794Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-08-11T16:05:23.901Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usergv0001 HTTP/1.1 SC="500" 2021-08-11T16:05:23.916Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=bf49ec9f-5534-4de4-9c81-7e37836027c7 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/user ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:23.915926Z InvocationID=2769ccac-0e0d-4677-ab85-c29f6e73dda5 LogTimestamp=2021-08-11T16:05:23.916011Z Entering 2021-08-11T16:05:23.916Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=bf49ec9f-5534-4de4-9c81-7e37836027c7 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/user ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:23.915926Z InvocationID=2769ccac-0e0d-4677-ab85-c29f6e73dda5 LogTimestamp=2021-08-11T16:05:23.916011Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/user HTTP/1.1 2021-08-11T16:05:23.933Z [qtp2045766957-18] ERROR o.o.sdc.be.servlets.UserAdminServlet RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/user EntryTimestamp=2021-08-11T16:05:23.916Z InvocationID=bc971006-2c19-4d64-9bfa-52954a5a81eb PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@30ebe', 'User [firstName=Giuseppe, lastName=Verdi, userId=gv0001, email=gv0001@openecomp.org, role=GOVERNOR, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByActionStatusComponentException out of org.openecomp.sdc.be.user.UserBusinessLogic#validateRole[157] in 15ms 2021-08-11T16:05:23.934Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=400 RequestID=null ElapsedTime=17 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/user ErrorDesc=Bad Request EntryTimestamp=2021-08-11T16:05:23.916Z InvocationID=bc971006-2c19-4d64-9bfa-52954a5a81eb ErrorCode=400 LogTimestamp=2021-08-11T16:05:23.933768Z ResponseDesc=Bad Request Exiting. 2021-08-11T16:05:23.934Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="400" 2021-08-11T16:05:23.949Z [qtp2045766957-15] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:23.949796Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-08-11T16:05:23.949Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:23.949796Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-08-11T16:05:23.950Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userjh0003 HTTP/1.1 SC="500" 2021-08-11T16:05:23.970Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=06e12792-5850-4bff-a2ca-46057d36e741 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/user ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:23.970360Z InvocationID=6944e844-6590-43d9-94b3-49b5edf3afae LogTimestamp=2021-08-11T16:05:23.970459Z Entering 2021-08-11T16:05:23.970Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=06e12792-5850-4bff-a2ca-46057d36e741 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/user ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:23.970360Z InvocationID=6944e844-6590-43d9-94b3-49b5edf3afae LogTimestamp=2021-08-11T16:05:23.970459Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/user HTTP/1.1 2021-08-11T16:05:23.984Z [qtp2045766957-14] ERROR o.o.sdc.be.servlets.UserAdminServlet AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4570: Error: An administrator is not allowed to change his/her role." RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/user EntryTimestamp=2021-08-11T16:05:23.971Z InvocationID=7c12b4ef-ec22-40bc-9079-c9b433ce9966 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p #createUser('Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/user)@30ebe', 'User [firstName=Jimmy, lastName=Hendrix, userId=jh0003, email=jh0003@openecomp.org, role=Admin, last login time=null]', 'jh0003'): thrown org.openecomp.sdc.be.components.impl.exceptions.ByResponseFormatComponentException out of org.openecomp.sdc.be.impl.ComponentsUtils#auditAdminUserActionAndThrowException[1113] in 12ms 2021-08-11T16:05:23.995Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "" STATUS = "409" DESC = "SVC4570: Error: An administrator is not allowed to change his/her role." ResponseCode=409 RequestID=null ElapsedTime=23 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/user ErrorDesc=Conflict EntryTimestamp=2021-08-11T16:05:23.971Z InvocationID=7c12b4ef-ec22-40bc-9079-c9b433ce9966 ErrorCode=409 LogTimestamp=2021-08-11T16:05:23.994955Z ResponseDesc=Conflict Exiting. 2021-08-11T16:05:23.996Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="409" 2021-08-11T16:05:24.012Z [qtp2045766957-20] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:24.011957Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-08-11T16:05:24.012Z [qtp2045766957-20] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:24.011957Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-08-11T16:05:24.012Z [qtp2045766957-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userjm0007 HTTP/1.1 SC="500" 2021-08-11T16:05:24.026Z [qtp2045766957-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=54ad252d-ffed-4782-9f62-ac0d8e0e03ae InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/user ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:24.026094Z InvocationID=5ba759ba-11a0-4fbc-8f97-2276c73a2427 LogTimestamp=2021-08-11T16:05:24.026175Z Entering 2021-08-11T16:05:24.026Z [qtp2045766957-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=54ad252d-ffed-4782-9f62-ac0d8e0e03ae InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/user ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:24.026094Z InvocationID=5ba759ba-11a0-4fbc-8f97-2276c73a2427 LogTimestamp=2021-08-11T16:05:24.026175Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/user HTTP/1.1 2021-08-11T16:05:25.265Z [qtp2045766957-11] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "jm0007 Joni Mitchell jm0007@openecomp.org TESTER" STATUS = "201" DESC = "OK" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/user EntryTimestamp=2021-08-11T16:05:24.026Z InvocationID=1e5547ca-7cfb-44f9-84a5-21b36931cf64 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:05:25.266Z [qtp2045766957-11] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "jm0007 Joni Mitchell jm0007@openecomp.org TESTER" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/user TargetServiceName=null EntryTimestamp=2021-08-11T16:05:24.026Z InvocationID=1e5547ca-7cfb-44f9-84a5-21b36931cf64 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:05:25.267Z [qtp2045766957-11] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "jm0007 Joni Mitchell jm0007@openecomp.org TESTER" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=1240 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/user TargetServiceName=null EntryTimestamp=2021-08-11T16:05:24.026Z InvocationID=1e5547ca-7cfb-44f9-84a5-21b36931cf64 ErrorCode=500 LogTimestamp=2021-08-11T16:05:25.266870Z ResponseDesc=Created Exiting. 2021-08-11T16:05:25.267Z [qtp2045766957-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="201" 2021-08-11T16:05:25.292Z [qtp2045766957-12] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:25.292519Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-08-11T16:05:25.292Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:25.292519Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-08-11T16:05:25.293Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercs0008 HTTP/1.1 SC="500" 2021-08-11T16:05:25.310Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=8f4cdbfc-315a-4a2e-936f-a5050cf0559b InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/user ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:25.309724Z InvocationID=9927ed0b-4dbe-466f-9d3b-e15ceeb05689 LogTimestamp=2021-08-11T16:05:25.309808Z Entering 2021-08-11T16:05:25.310Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=8f4cdbfc-315a-4a2e-936f-a5050cf0559b InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/user ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:25.309724Z InvocationID=9927ed0b-4dbe-466f-9d3b-e15ceeb05689 LogTimestamp=2021-08-11T16:05:25.309808Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/user HTTP/1.1 2021-08-11T16:05:25.750Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=19478f42-915f-4c06-8aae-5680aa2e1bc4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:05:25.751Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=19478f42-915f-4c06-8aae-5680aa2e1bc4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@d6171be 2021-08-11T16:05:25.751Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=19478f42-915f-4c06-8aae-5680aa2e1bc4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:05:25.751Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=19478f42-915f-4c06-8aae-5680aa2e1bc4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:05:25.807Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=14607cf0-b130-4b91-a290-ef0ae1091310 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=cds ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.146.3 StatusCode=INPROGRESS RemoteHost=10.242.146.3 ServiceName=v1/artifactTypes ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:25.806617Z InvocationID=80fa1ef0-052a-4c93-b59a-13a6984a15bb LogTimestamp=2021-08-11T16:05:25.807114Z Entering 2021-08-11T16:05:25.807Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=14607cf0-b130-4b91-a290-ef0ae1091310 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=cds ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.146.3 StatusCode=INPROGRESS userId=null RemoteHost=10.242.146.3 ServiceName=v1/artifactTypes ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:25.806617Z InvocationID=80fa1ef0-052a-4c93-b59a-13a6984a15bb LogTimestamp=2021-08-11T16:05:25.807114Z localAddr=10.242.24.223 remoteAddr=10.242.146.3 GET /sdc/v1/artifactTypes HTTP/1.1 2021-08-11T16:05:25.809Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=2 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.8) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/artifactTypes EntryTimestamp=2021-08-11T16:05:25.807Z InvocationID=de3e218b-0482-443b-90d6-df063cba28c8 LogTimestamp=2021-08-11T16:05:25.809184Z ResponseDesc=OK Exiting. 2021-08-11T16:05:25.809Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc/v1/artifactTypes HTTP/1.1 SC="200" 2021-08-11T16:05:25.938Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=1cc7f096-8bb0-4cb7-9c2d-c5c7b81f7bb5 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=cds ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.146.3 StatusCode=INPROGRESS RemoteHost=10.242.146.3 ServiceName=v1/registerForDistribution ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:25.938151Z InvocationID=7429392e-4180-45a6-8e07-1b99f6102134 LogTimestamp=2021-08-11T16:05:25.938262Z Entering 2021-08-11T16:05:25.938Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=1cc7f096-8bb0-4cb7-9c2d-c5c7b81f7bb5 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=cds ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.146.3 StatusCode=INPROGRESS userId=null RemoteHost=10.242.146.3 ServiceName=v1/registerForDistribution ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:25.938151Z InvocationID=7429392e-4180-45a6-8e07-1b99f6102134 LogTimestamp=2021-08-11T16:05:25.938262Z localAddr=10.242.24.223 remoteAddr=10.242.146.3 POST /sdc/v1/registerForDistribution HTTP/1.1 2021-08-11T16:05:25.948Z [qtp2045766957-14] WARN c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:05:25.939Z InvocationID=4c95974d-3e13-4eae-bd4b-9a7ea7621d1c PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.8) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/producers/vVXZrlAOjp2PoZT0 will send credentials over a clear channel. 2021-08-11T16:05:25.948Z [qtp2045766957-14] INFO c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:05:25.939Z InvocationID=4c95974d-3e13-4eae-bd4b-9a7ea7621d1c PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.8) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/producers/vVXZrlAOjp2PoZT0 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:05:25.974Z [qtp2045766957-14] INFO c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:05:25.939Z InvocationID=4c95974d-3e13-4eae-bd4b-9a7ea7621d1c PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.8) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p \t--> HTTP/1.1 200 OK 2021-08-11T16:05:25.979Z [qtp2045766957-14] WARN c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "vVXZrlAOjp2PoZT0" STATUS = "200" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:05:25.939Z InvocationID=4c95974d-3e13-4eae-bd4b-9a7ea7621d1c PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.8) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/consumers/vVXZrlAOjp2PoZT0 will send credentials over a clear channel. 2021-08-11T16:05:25.979Z [qtp2045766957-14] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "vVXZrlAOjp2PoZT0" STATUS = "200" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:05:25.939Z InvocationID=4c95974d-3e13-4eae-bd4b-9a7ea7621d1c PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.8) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/consumers/vVXZrlAOjp2PoZT0 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:05:26.035Z [qtp2045766957-14] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "vVXZrlAOjp2PoZT0" STATUS = "200" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:05:25.939Z InvocationID=4c95974d-3e13-4eae-bd4b-9a7ea7621d1c PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.8) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p \t--> HTTP/1.1 200 OK 2021-08-11T16:05:26.043Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "DRegister" CONSUMER_ID = "cds" API_KEY = "vVXZrlAOjp2PoZT0" D_ENV = "AUTO" STATUS = "200" DESC = "" DNOTIF_TOPIC = "SDC-DISTR-NOTIF-TOPIC-AUTO" DSTATUS_TOPIC = "SDC-DISTR-STATUS-TOPIC-AUTO" ResponseCode=200 RequestID=null ElapsedTime=104 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.8) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:05:25.939Z InvocationID=4c95974d-3e13-4eae-bd4b-9a7ea7621d1c LogTimestamp=2021-08-11T16:05:26.043603Z ResponseDesc=OK Exiting. 2021-08-11T16:05:26.044Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc/v1/registerForDistribution HTTP/1.1 SC="200" 2021-08-11T16:05:26.676Z [qtp2045766957-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "cs0008 Carlos Santana cs0008r@openecomp.org DESIGNER" STATUS = "201" DESC = "OK" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/user EntryTimestamp=2021-08-11T16:05:25.310Z InvocationID=46243f72-5ddf-4c9b-adb9-5ceb4d6b9830 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:05:26.676Z [qtp2045766957-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "cs0008 Carlos Santana cs0008r@openecomp.org DESIGNER" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/user TargetServiceName=null EntryTimestamp=2021-08-11T16:05:25.310Z InvocationID=46243f72-5ddf-4c9b-adb9-5ceb4d6b9830 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:05:26.677Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddUser" MODIFIER = "Jimmy Hendrix(jh0003)" USER_BEFORE = "" USER_AFTER = "cs0008 Carlos Santana cs0008r@openecomp.org DESIGNER" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=1367 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/user TargetServiceName=null EntryTimestamp=2021-08-11T16:05:25.310Z InvocationID=46243f72-5ddf-4c9b-adb9-5ceb4d6b9830 ErrorCode=500 LogTimestamp=2021-08-11T16:05:26.677026Z ResponseDesc=Created Exiting. 2021-08-11T16:05:26.677Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/user HTTP/1.1 SC="201" 2021-08-11T16:05:26.855Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=165ce735-2c08-49a6-b881-b671045366be InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:26.855484Z InvocationID=ea53ae8d-1a84-4454-853b-7c0a5e0bae7f LogTimestamp=2021-08-11T16:05:26.855591Z Entering 2021-08-11T16:05:26.856Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=165ce735-2c08-49a6-b881-b671045366be InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:26.855484Z InvocationID=ea53ae8d-1a84-4454-853b-7c0a5e0bae7f LogTimestamp=2021-08-11T16:05:26.855591Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-08-11T16:05:26.870Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=14 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/user/jh0003 EntryTimestamp=2021-08-11T16:05:26.856Z InvocationID=3a7e1e72-9dab-4de3-8720-1116cf63c9d8 LogTimestamp=2021-08-11T16:05:26.870004Z ResponseDesc=OK Exiting. 2021-08-11T16:05:26.870Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2021-08-11T16:05:26.953Z [qtp2045766957-14] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:26.953016Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-08-11T16:05:26.953Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:26.953016Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-08-11T16:05:26.954Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/useraai HTTP/1.1 SC="500" 2021-08-11T16:05:26.994Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=1b5cad9c-94ea-430f-8457-3ae58ef321c2 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/consumers ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:26.994542Z InvocationID=7f5b26f8-42e3-41ba-aaf7-d7dfb5b4ef46 LogTimestamp=2021-08-11T16:05:26.994631Z Entering 2021-08-11T16:05:26.995Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=1b5cad9c-94ea-430f-8457-3ae58ef321c2 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/consumers ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:26.994542Z InvocationID=7f5b26f8-42e3-41ba-aaf7-d7dfb5b4ef46 LogTimestamp=2021-08-11T16:05:26.994631Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-08-11T16:05:27.072Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers EntryTimestamp=2021-08-11T16:05:26.995Z InvocationID=8d963c3d-e5f3-4847-b44c-8afd2ddcd9bf PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id aai 2021-08-11T16:05:27.914Z [qtp2045766957-17] ERROR o.o.s.b.m.o.impl.ConsumerOperation RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:26.995Z InvocationID=8d963c3d-e5f3-4847-b44c-8afd2ddcd9bf ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null Error returned after get Consumer Data node aai. status returned is NOT_FOUND 2021-08-11T16:05:28.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a91e187-52da-477b-bb6e-8ae12770b6f3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:05:28.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a91e187-52da-477b-bb6e-8ae12770b6f3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@42b63979 2021-08-11T16:05:28.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a91e187-52da-477b-bb6e-8ae12770b6f3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:05:28.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a91e187-52da-477b-bb6e-8ae12770b6f3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:05:29.654Z [qtp2045766957-17] WARN o.o.s.b.a.impl.AuditingManager AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "aai,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:26.995Z InvocationID=8d963c3d-e5f3-4847-b44c-8afd2ddcd9bf ErrorCode=500 Failed to persist to cassandra auditing event: GENERAL_ERROR 2021-08-11T16:05:30.212Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "aai,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=3217 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:26.995Z InvocationID=8d963c3d-e5f3-4847-b44c-8afd2ddcd9bf ErrorCode=500 LogTimestamp=2021-08-11T16:05:30.212089Z ResponseDesc=Created Exiting. 2021-08-11T16:05:30.212Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-08-11T16:05:30.242Z [qtp2045766957-20] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:30.242749Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-08-11T16:05:30.243Z [qtp2045766957-20] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:30.242749Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-08-11T16:05:30.243Z [qtp2045766957-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userappc HTTP/1.1 SC="500" 2021-08-11T16:05:30.260Z [qtp2045766957-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=1e0b3d40-e10e-4843-b74a-bc0988337c23 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/consumers ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:30.259819Z InvocationID=96020639-af8a-4de3-90e0-2226bf861def LogTimestamp=2021-08-11T16:05:30.259914Z Entering 2021-08-11T16:05:30.260Z [qtp2045766957-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=1e0b3d40-e10e-4843-b74a-bc0988337c23 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/consumers ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:30.259819Z InvocationID=96020639-af8a-4de3-90e0-2226bf861def LogTimestamp=2021-08-11T16:05:30.259914Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-08-11T16:05:30.280Z [qtp2045766957-20] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers EntryTimestamp=2021-08-11T16:05:30.261Z InvocationID=5f408070-56d1-407b-828b-a8d3aed27158 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id appc 2021-08-11T16:05:30.830Z [qtp2045766957-20] ERROR o.o.s.b.m.o.impl.ConsumerOperation RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:30.261Z InvocationID=5f408070-56d1-407b-828b-a8d3aed27158 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null Error returned after get Consumer Data node appc. status returned is NOT_FOUND 2021-08-11T16:05:31.691Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b4fd9af3-e13c-4d23-8d6b-a8ddaa217167 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:05:31.691Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b4fd9af3-e13c-4d23-8d6b-a8ddaa217167 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@f80ca58 2021-08-11T16:05:31.691Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b4fd9af3-e13c-4d23-8d6b-a8ddaa217167 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:05:31.691Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b4fd9af3-e13c-4d23-8d6b-a8ddaa217167 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:05:32.067Z [qtp2045766957-20] WARN o.o.s.b.a.impl.AuditingManager AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "appc,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:30.261Z InvocationID=5f408070-56d1-407b-828b-a8d3aed27158 ErrorCode=500 Failed to persist to cassandra auditing event: GENERAL_ERROR 2021-08-11T16:05:32.620Z [qtp2045766957-20] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "appc,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=2358 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:30.261Z InvocationID=5f408070-56d1-407b-828b-a8d3aed27158 ErrorCode=500 LogTimestamp=2021-08-11T16:05:32.619981Z ResponseDesc=Created Exiting. 2021-08-11T16:05:32.621Z [qtp2045766957-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-08-11T16:05:32.638Z [qtp2045766957-11] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:32.638037Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-08-11T16:05:32.638Z [qtp2045766957-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:32.638037Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-08-11T16:05:32.638Z [qtp2045766957-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userdcae HTTP/1.1 SC="500" 2021-08-11T16:05:32.653Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=990eabe0-b9f0-4f35-b4da-c1f6b5d6793e InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/consumers ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:32.652941Z InvocationID=164247e6-68b2-4352-8a03-6e32687d9a38 LogTimestamp=2021-08-11T16:05:32.653045Z Entering 2021-08-11T16:05:32.653Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=990eabe0-b9f0-4f35-b4da-c1f6b5d6793e InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/consumers ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:32.652941Z InvocationID=164247e6-68b2-4352-8a03-6e32687d9a38 LogTimestamp=2021-08-11T16:05:32.653045Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-08-11T16:05:32.663Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers EntryTimestamp=2021-08-11T16:05:32.654Z InvocationID=c966dc82-acf8-425d-a24d-6f9c2b66fc00 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id dcae 2021-08-11T16:05:33.187Z [qtp2045766957-18] ERROR o.o.s.b.m.o.impl.ConsumerOperation RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:32.654Z InvocationID=c966dc82-acf8-425d-a24d-6f9c2b66fc00 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null Error returned after get Consumer Data node dcae. status returned is NOT_FOUND 2021-08-11T16:05:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:05:33.332Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:05:33.710Z [qtp2045766957-18] WARN o.o.s.b.a.impl.AuditingManager AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "dcae,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:32.654Z InvocationID=c966dc82-acf8-425d-a24d-6f9c2b66fc00 ErrorCode=500 Failed to persist to cassandra auditing event: GENERAL_ERROR 2021-08-11T16:05:34.271Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "dcae,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=1617 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:32.654Z InvocationID=c966dc82-acf8-425d-a24d-6f9c2b66fc00 ErrorCode=500 LogTimestamp=2021-08-11T16:05:34.271021Z ResponseDesc=Created Exiting. 2021-08-11T16:05:34.272Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-08-11T16:05:34.287Z [qtp2045766957-18] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:34.287144Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-08-11T16:05:34.287Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:34.287144Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-08-11T16:05:34.288Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermso HTTP/1.1 SC="500" 2021-08-11T16:05:34.310Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=4cdd400d-958c-45f5-8e98-7f91a9d0cf6d InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/consumers ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:34.309974Z InvocationID=c4e565e0-c7d4-4d27-87c8-7ab10679d89b LogTimestamp=2021-08-11T16:05:34.310132Z Entering 2021-08-11T16:05:34.310Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=4cdd400d-958c-45f5-8e98-7f91a9d0cf6d InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/consumers ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:34.309974Z InvocationID=c4e565e0-c7d4-4d27-87c8-7ab10679d89b LogTimestamp=2021-08-11T16:05:34.310132Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-08-11T16:05:34.323Z [qtp2045766957-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers EntryTimestamp=2021-08-11T16:05:34.311Z InvocationID=6798e09c-6d3a-43cd-8269-f0f745de9a4a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id mso 2021-08-11T16:05:34.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0dca379-b88d-410c-9a28-0d369260ebb2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:05:34.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0dca379-b88d-410c-9a28-0d369260ebb2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@163ded1 2021-08-11T16:05:34.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0dca379-b88d-410c-9a28-0d369260ebb2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:05:34.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0dca379-b88d-410c-9a28-0d369260ebb2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:05:34.851Z [qtp2045766957-15] ERROR o.o.s.b.m.o.impl.ConsumerOperation RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:34.311Z InvocationID=6798e09c-6d3a-43cd-8269-f0f745de9a4a ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null Error returned after get Consumer Data node mso. status returned is NOT_FOUND 2021-08-11T16:05:35.369Z [qtp2045766957-15] WARN o.o.s.b.a.impl.AuditingManager AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "mso,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:34.311Z InvocationID=6798e09c-6d3a-43cd-8269-f0f745de9a4a ErrorCode=500 Failed to persist to cassandra auditing event: GENERAL_ERROR 2021-08-11T16:05:35.953Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "mso,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=1641 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:34.311Z InvocationID=6798e09c-6d3a-43cd-8269-f0f745de9a4a ErrorCode=500 LogTimestamp=2021-08-11T16:05:35.952837Z ResponseDesc=Created Exiting. 2021-08-11T16:05:35.954Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-08-11T16:05:35.979Z [qtp2045766957-15] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:35.979573Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-08-11T16:05:35.979Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:35.979573Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-08-11T16:05:35.980Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usersdnc HTTP/1.1 SC="500" 2021-08-11T16:05:36.002Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=f1adcc61-2565-4b82-a38d-7ff890b83742 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/consumers ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:36.001432Z InvocationID=a9816378-a31e-4176-917b-45f4a49f9de2 LogTimestamp=2021-08-11T16:05:36.001579Z Entering 2021-08-11T16:05:36.002Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=f1adcc61-2565-4b82-a38d-7ff890b83742 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/consumers ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:36.001432Z InvocationID=a9816378-a31e-4176-917b-45f4a49f9de2 LogTimestamp=2021-08-11T16:05:36.001579Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-08-11T16:05:36.025Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers EntryTimestamp=2021-08-11T16:05:36.002Z InvocationID=38375af0-5c42-4850-b072-90a351a16fc5 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id sdnc 2021-08-11T16:05:36.563Z [qtp2045766957-12] ERROR o.o.s.b.m.o.impl.ConsumerOperation RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:36.002Z InvocationID=38375af0-5c42-4850-b072-90a351a16fc5 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null Error returned after get Consumer Data node sdnc. status returned is NOT_FOUND 2021-08-11T16:05:37.079Z [qtp2045766957-12] WARN o.o.s.b.a.impl.AuditingManager AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "sdnc,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:36.002Z InvocationID=38375af0-5c42-4850-b072-90a351a16fc5 ErrorCode=500 Failed to persist to cassandra auditing event: GENERAL_ERROR 2021-08-11T16:05:37.751Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55888a1e-5bbc-4771-96f5-cd38ce56338e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:05:37.751Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55888a1e-5bbc-4771-96f5-cd38ce56338e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@789a3c2 2021-08-11T16:05:37.751Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55888a1e-5bbc-4771-96f5-cd38ce56338e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:05:37.751Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55888a1e-5bbc-4771-96f5-cd38ce56338e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:05:37.761Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "sdnc,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=1758 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:36.002Z InvocationID=38375af0-5c42-4850-b072-90a351a16fc5 ErrorCode=500 LogTimestamp=2021-08-11T16:05:37.760614Z ResponseDesc=Created Exiting. 2021-08-11T16:05:37.761Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-08-11T16:05:37.776Z [qtp2045766957-15] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:37.776701Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-08-11T16:05:37.776Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:37.776701Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-08-11T16:05:37.777Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/uservid HTTP/1.1 SC="500" 2021-08-11T16:05:37.855Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=5008c141-941f-4b72-832a-c7cdb6343e80 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/consumers ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:37.854639Z InvocationID=a9bb5ec4-4639-461b-abb9-95e24e3ff932 LogTimestamp=2021-08-11T16:05:37.854767Z Entering 2021-08-11T16:05:37.855Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=5008c141-941f-4b72-832a-c7cdb6343e80 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/consumers ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:37.854639Z InvocationID=a9bb5ec4-4639-461b-abb9-95e24e3ff932 LogTimestamp=2021-08-11T16:05:37.854767Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-08-11T16:05:37.863Z [qtp2045766957-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers EntryTimestamp=2021-08-11T16:05:37.855Z InvocationID=dce62ef0-b3f7-449a-b811-175d45620919 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id vid 2021-08-11T16:05:37.908Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=6cd05ab3-a507-40de-82e5-5ed39dd0d643 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=dcae ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.24.9 StatusCode=INPROGRESS RemoteHost=10.242.24.9 ServiceName=v1/artifactTypes ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:37.908178Z InvocationID=19afe582-a4e9-40e8-a9f9-cec0e610f1fb LogTimestamp=2021-08-11T16:05:37.908323Z Entering 2021-08-11T16:05:37.908Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=6cd05ab3-a507-40de-82e5-5ed39dd0d643 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=dcae ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.24.9 StatusCode=INPROGRESS userId=null RemoteHost=10.242.24.9 ServiceName=v1/artifactTypes ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:37.908178Z InvocationID=19afe582-a4e9-40e8-a9f9-cec0e610f1fb LogTimestamp=2021-08-11T16:05:37.908323Z localAddr=10.242.24.223 remoteAddr=10.242.24.9 GET /sdc/v1/artifactTypes HTTP/1.1 2021-08-11T16:05:37.912Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=2 PartnerName=Apache-HttpClient/4.5.7 (Java/11.0.6) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/artifactTypes EntryTimestamp=2021-08-11T16:05:37.909Z InvocationID=31e5163e-478d-4325-8771-88f68dcfff17 LogTimestamp=2021-08-11T16:05:37.911737Z ResponseDesc=OK Exiting. 2021-08-11T16:05:37.912Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc/v1/artifactTypes HTTP/1.1 SC="200" 2021-08-11T16:05:38.004Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=3df7b6d5-fd4a-4c25-a4ab-f03bb4d1fd10 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=dcae ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.24.9 StatusCode=INPROGRESS RemoteHost=10.242.24.9 ServiceName=v1/registerForDistribution ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:38.003808Z InvocationID=e1c4acb9-f841-4d92-b417-176462aeb504 LogTimestamp=2021-08-11T16:05:38.003951Z Entering 2021-08-11T16:05:38.004Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=3df7b6d5-fd4a-4c25-a4ab-f03bb4d1fd10 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=dcae ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.24.9 StatusCode=INPROGRESS userId=null RemoteHost=10.242.24.9 ServiceName=v1/registerForDistribution ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:38.003808Z InvocationID=e1c4acb9-f841-4d92-b417-176462aeb504 LogTimestamp=2021-08-11T16:05:38.003951Z localAddr=10.242.24.223 remoteAddr=10.242.24.9 POST /sdc/v1/registerForDistribution HTTP/1.1 2021-08-11T16:05:38.010Z [qtp2045766957-12] WARN c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:05:38.004Z InvocationID=9d194717-b505-4041-a967-8fb741405fd6 PartnerName=Apache-HttpClient/4.5.7 (Java/11.0.6) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/producers/0NAh3GGRI8mBajPr will send credentials over a clear channel. 2021-08-11T16:05:38.010Z [qtp2045766957-12] INFO c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:05:38.004Z InvocationID=9d194717-b505-4041-a967-8fb741405fd6 PartnerName=Apache-HttpClient/4.5.7 (Java/11.0.6) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/producers/0NAh3GGRI8mBajPr (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:05:38.031Z [qtp2045766957-12] INFO c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:05:38.004Z InvocationID=9d194717-b505-4041-a967-8fb741405fd6 PartnerName=Apache-HttpClient/4.5.7 (Java/11.0.6) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p \t--> HTTP/1.1 200 OK 2021-08-11T16:05:38.036Z [qtp2045766957-12] WARN c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "0NAh3GGRI8mBajPr" STATUS = "200" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:05:38.004Z InvocationID=9d194717-b505-4041-a967-8fb741405fd6 PartnerName=Apache-HttpClient/4.5.7 (Java/11.0.6) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/consumers/0NAh3GGRI8mBajPr will send credentials over a clear channel. 2021-08-11T16:05:38.036Z [qtp2045766957-12] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "0NAh3GGRI8mBajPr" STATUS = "200" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:05:38.004Z InvocationID=9d194717-b505-4041-a967-8fb741405fd6 PartnerName=Apache-HttpClient/4.5.7 (Java/11.0.6) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/consumers/0NAh3GGRI8mBajPr (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:05:38.060Z [qtp2045766957-12] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "0NAh3GGRI8mBajPr" STATUS = "200" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:05:38.004Z InvocationID=9d194717-b505-4041-a967-8fb741405fd6 PartnerName=Apache-HttpClient/4.5.7 (Java/11.0.6) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p \t--> HTTP/1.1 200 OK 2021-08-11T16:05:38.072Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "DRegister" CONSUMER_ID = "dcae-sch" API_KEY = "0NAh3GGRI8mBajPr" D_ENV = "AUTO" STATUS = "200" DESC = "" DNOTIF_TOPIC = "SDC-DISTR-NOTIF-TOPIC-AUTO" DSTATUS_TOPIC = "SDC-DISTR-STATUS-TOPIC-AUTO" ResponseCode=200 RequestID=null ElapsedTime=68 PartnerName=Apache-HttpClient/4.5.7 (Java/11.0.6) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:05:38.004Z InvocationID=9d194717-b505-4041-a967-8fb741405fd6 LogTimestamp=2021-08-11T16:05:38.072186Z ResponseDesc=OK Exiting. 2021-08-11T16:05:38.073Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc/v1/registerForDistribution HTTP/1.1 SC="200" 2021-08-11T16:05:39.167Z [qtp2045766957-15] ERROR o.o.s.b.m.o.impl.ConsumerOperation RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:37.855Z InvocationID=dce62ef0-b3f7-449a-b811-175d45620919 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null Error returned after get Consumer Data node vid. status returned is NOT_FOUND 2021-08-11T16:05:39.327Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=c092d335-6305-4e8d-8161-23b36bfd4f9a InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=aai ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.129 StatusCode=INPROGRESS RemoteHost=10.242.110.129 ServiceName=v1/artifactTypes ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:39.327036Z InvocationID=28bfb0a8-c68a-45e4-a72a-52fd34427d2c LogTimestamp=2021-08-11T16:05:39.327325Z Entering 2021-08-11T16:05:39.328Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=c092d335-6305-4e8d-8161-23b36bfd4f9a InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=aai ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.129 StatusCode=INPROGRESS userId=null RemoteHost=10.242.110.129 ServiceName=v1/artifactTypes ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:39.327036Z InvocationID=28bfb0a8-c68a-45e4-a72a-52fd34427d2c LogTimestamp=2021-08-11T16:05:39.327325Z localAddr=10.242.24.223 remoteAddr=10.242.110.129 GET /sdc/v1/artifactTypes HTTP/1.1 2021-08-11T16:05:39.330Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=2 PartnerName=Apache-HttpClient/4.5.10 (Java/11.0.9) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/artifactTypes EntryTimestamp=2021-08-11T16:05:39.328Z InvocationID=0c7a3c05-6da2-4eb9-9e50-3eb96badc919 LogTimestamp=2021-08-11T16:05:39.330516Z ResponseDesc=OK Exiting. 2021-08-11T16:05:39.331Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc/v1/artifactTypes HTTP/1.1 SC="200" 2021-08-11T16:05:39.440Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=a3f2b3b0-8231-4002-9cc7-97cc2fed74ca InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=aai ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.129 StatusCode=INPROGRESS RemoteHost=10.242.110.129 ServiceName=v1/registerForDistribution ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:39.440014Z InvocationID=1b2562e6-f7d0-4177-b22f-acbc3d3873fa LogTimestamp=2021-08-11T16:05:39.440092Z Entering 2021-08-11T16:05:39.440Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=a3f2b3b0-8231-4002-9cc7-97cc2fed74ca InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=aai ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.129 StatusCode=INPROGRESS userId=null RemoteHost=10.242.110.129 ServiceName=v1/registerForDistribution ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:39.440014Z InvocationID=1b2562e6-f7d0-4177-b22f-acbc3d3873fa LogTimestamp=2021-08-11T16:05:39.440092Z localAddr=10.242.24.223 remoteAddr=10.242.110.129 POST /sdc/v1/registerForDistribution HTTP/1.1 2021-08-11T16:05:39.445Z [qtp2045766957-14] WARN c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:05:39.440Z InvocationID=a757fa12-715e-477f-a876-e3ab39195c06 PartnerName=Apache-HttpClient/4.5.10 (Java/11.0.9) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/producers/KtQYtWgxMoHcM7kM will send credentials over a clear channel. 2021-08-11T16:05:39.445Z [qtp2045766957-14] INFO c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:05:39.440Z InvocationID=a757fa12-715e-477f-a876-e3ab39195c06 PartnerName=Apache-HttpClient/4.5.10 (Java/11.0.9) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/producers/KtQYtWgxMoHcM7kM (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:05:39.462Z [qtp2045766957-14] INFO c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:05:39.440Z InvocationID=a757fa12-715e-477f-a876-e3ab39195c06 PartnerName=Apache-HttpClient/4.5.10 (Java/11.0.9) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p \t--> HTTP/1.1 200 OK 2021-08-11T16:05:39.467Z [qtp2045766957-14] WARN c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "KtQYtWgxMoHcM7kM" STATUS = "200" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:05:39.440Z InvocationID=a757fa12-715e-477f-a876-e3ab39195c06 PartnerName=Apache-HttpClient/4.5.10 (Java/11.0.9) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/consumers/KtQYtWgxMoHcM7kM will send credentials over a clear channel. 2021-08-11T16:05:39.467Z [qtp2045766957-14] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "KtQYtWgxMoHcM7kM" STATUS = "200" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:05:39.440Z InvocationID=a757fa12-715e-477f-a876-e3ab39195c06 PartnerName=Apache-HttpClient/4.5.10 (Java/11.0.9) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/consumers/KtQYtWgxMoHcM7kM (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:05:39.482Z [qtp2045766957-14] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "KtQYtWgxMoHcM7kM" STATUS = "200" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:05:39.440Z InvocationID=a757fa12-715e-477f-a876-e3ab39195c06 PartnerName=Apache-HttpClient/4.5.10 (Java/11.0.9) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p \t--> HTTP/1.1 200 OK 2021-08-11T16:05:39.548Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "DRegister" CONSUMER_ID = "aai-ml" API_KEY = "KtQYtWgxMoHcM7kM" D_ENV = "AUTO" STATUS = "200" DESC = "" DNOTIF_TOPIC = "SDC-DISTR-NOTIF-TOPIC-AUTO" DSTATUS_TOPIC = "SDC-DISTR-STATUS-TOPIC-AUTO" ResponseCode=200 RequestID=null ElapsedTime=107 PartnerName=Apache-HttpClient/4.5.10 (Java/11.0.9) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:05:39.440Z InvocationID=a757fa12-715e-477f-a876-e3ab39195c06 LogTimestamp=2021-08-11T16:05:39.547920Z ResponseDesc=OK Exiting. 2021-08-11T16:05:39.548Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc/v1/registerForDistribution HTTP/1.1 SC="200" 2021-08-11T16:05:39.755Z [qtp2045766957-15] WARN o.o.s.b.a.impl.AuditingManager AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "vid,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:37.855Z InvocationID=dce62ef0-b3f7-449a-b811-175d45620919 ErrorCode=500 Failed to persist to cassandra auditing event: GENERAL_ERROR 2021-08-11T16:05:40.376Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "vid,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=2521 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:37.855Z InvocationID=dce62ef0-b3f7-449a-b811-175d45620919 ErrorCode=500 LogTimestamp=2021-08-11T16:05:40.376253Z ResponseDesc=Created Exiting. 2021-08-11T16:05:40.377Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-08-11T16:05:40.392Z [qtp2045766957-20] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:40.392256Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-08-11T16:05:40.392Z [qtp2045766957-20] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:40.392256Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-08-11T16:05:40.393Z [qtp2045766957-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercognita HTTP/1.1 SC="500" 2021-08-11T16:05:40.409Z [qtp2045766957-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=f8e2c987-849d-490c-a7fe-934e178b7f9c InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/consumers ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:40.409060Z InvocationID=f2387e5e-8a7a-4820-b96d-bf5356f90bdc LogTimestamp=2021-08-11T16:05:40.409136Z Entering 2021-08-11T16:05:40.409Z [qtp2045766957-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=f8e2c987-849d-490c-a7fe-934e178b7f9c InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/consumers ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:40.409060Z InvocationID=f2387e5e-8a7a-4820-b96d-bf5356f90bdc LogTimestamp=2021-08-11T16:05:40.409136Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-08-11T16:05:40.456Z [qtp2045766957-11] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers EntryTimestamp=2021-08-11T16:05:40.409Z InvocationID=0d1c3705-61c9-42d2-b723-32e985b25343 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id cognita 2021-08-11T16:05:40.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a97c7da-e2c5-4225-a15c-0061c607790f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:05:40.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a97c7da-e2c5-4225-a15c-0061c607790f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@74873fbb 2021-08-11T16:05:40.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a97c7da-e2c5-4225-a15c-0061c607790f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:05:40.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a97c7da-e2c5-4225-a15c-0061c607790f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:05:41.057Z [qtp2045766957-11] ERROR o.o.s.b.m.o.impl.ConsumerOperation RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:40.409Z InvocationID=0d1c3705-61c9-42d2-b723-32e985b25343 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null Error returned after get Consumer Data node cognita. status returned is NOT_FOUND 2021-08-11T16:05:41.580Z [qtp2045766957-11] WARN o.o.s.b.a.impl.AuditingManager AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "cognita,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:40.409Z InvocationID=0d1c3705-61c9-42d2-b723-32e985b25343 ErrorCode=500 Failed to persist to cassandra auditing event: GENERAL_ERROR 2021-08-11T16:05:42.167Z [qtp2045766957-11] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "cognita,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=1757 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:40.409Z InvocationID=0d1c3705-61c9-42d2-b723-32e985b25343 ErrorCode=500 LogTimestamp=2021-08-11T16:05:42.166811Z ResponseDesc=Created Exiting. 2021-08-11T16:05:42.168Z [qtp2045766957-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-08-11T16:05:42.199Z [qtp2045766957-11] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:42.194494Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-08-11T16:05:42.199Z [qtp2045766957-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:42.194494Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-08-11T16:05:42.208Z [qtp2045766957-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userclamp HTTP/1.1 SC="500" 2021-08-11T16:05:42.230Z [qtp2045766957-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=bc6c6b59-0f37-4233-9dad-4b952fc3d28c InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/consumers ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:42.230303Z InvocationID=8cff8c0d-d3a2-4da7-a0b9-b8ed9399a896 LogTimestamp=2021-08-11T16:05:42.230540Z Entering 2021-08-11T16:05:42.231Z [qtp2045766957-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=bc6c6b59-0f37-4233-9dad-4b952fc3d28c InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/consumers ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:42.230303Z InvocationID=8cff8c0d-d3a2-4da7-a0b9-b8ed9399a896 LogTimestamp=2021-08-11T16:05:42.230540Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-08-11T16:05:42.259Z [qtp2045766957-11] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers EntryTimestamp=2021-08-11T16:05:42.231Z InvocationID=c2984f6f-8e30-4b43-94c5-1a1479029f55 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id clamp 2021-08-11T16:05:42.831Z [qtp2045766957-11] ERROR o.o.s.b.m.o.impl.ConsumerOperation RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:42.231Z InvocationID=c2984f6f-8e30-4b43-94c5-1a1479029f55 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null Error returned after get Consumer Data node clamp. status returned is NOT_FOUND 2021-08-11T16:05:43.359Z [qtp2045766957-11] WARN o.o.s.b.a.impl.AuditingManager AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "clamp,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:42.231Z InvocationID=c2984f6f-8e30-4b43-94c5-1a1479029f55 ErrorCode=500 Failed to persist to cassandra auditing event: GENERAL_ERROR 2021-08-11T16:05:43.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f108a21d-8fdb-41de-a5eb-d34fc31aa370 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:05:43.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f108a21d-8fdb-41de-a5eb-d34fc31aa370 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7d5cf979 2021-08-11T16:05:43.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f108a21d-8fdb-41de-a5eb-d34fc31aa370 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:05:43.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f108a21d-8fdb-41de-a5eb-d34fc31aa370 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:05:43.938Z [qtp2045766957-11] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "clamp,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=1706 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:42.231Z InvocationID=c2984f6f-8e30-4b43-94c5-1a1479029f55 ErrorCode=500 LogTimestamp=2021-08-11T16:05:43.937881Z ResponseDesc=Created Exiting. 2021-08-11T16:05:43.938Z [qtp2045766957-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-08-11T16:05:43.952Z [qtp2045766957-12] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:43.952522Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-08-11T16:05:43.952Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:43.952522Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-08-11T16:05:43.953Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/uservfc HTTP/1.1 SC="500" 2021-08-11T16:05:43.964Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=f15516bb-2eed-4133-9960-a42ec1c12c85 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/consumers ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:43.964536Z InvocationID=9b687b2b-b270-4761-91e0-360500cb3fba LogTimestamp=2021-08-11T16:05:43.964612Z Entering 2021-08-11T16:05:43.965Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=f15516bb-2eed-4133-9960-a42ec1c12c85 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/consumers ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:43.964536Z InvocationID=9b687b2b-b270-4761-91e0-360500cb3fba LogTimestamp=2021-08-11T16:05:43.964612Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-08-11T16:05:43.975Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers EntryTimestamp=2021-08-11T16:05:43.965Z InvocationID=1c3a8cbf-99dc-44d2-b0a6-b120577a8bb8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id vfc 2021-08-11T16:05:44.498Z [qtp2045766957-17] ERROR o.o.s.b.m.o.impl.ConsumerOperation RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:43.965Z InvocationID=1c3a8cbf-99dc-44d2-b0a6-b120577a8bb8 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null Error returned after get Consumer Data node vfc. status returned is NOT_FOUND 2021-08-11T16:05:45.023Z [qtp2045766957-17] WARN o.o.s.b.a.impl.AuditingManager AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "vfc,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:43.965Z InvocationID=1c3a8cbf-99dc-44d2-b0a6-b120577a8bb8 ErrorCode=500 Failed to persist to cassandra auditing event: GENERAL_ERROR 2021-08-11T16:05:45.592Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "vfc,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=1627 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:43.965Z InvocationID=1c3a8cbf-99dc-44d2-b0a6-b120577a8bb8 ErrorCode=500 LogTimestamp=2021-08-11T16:05:45.592012Z ResponseDesc=Created Exiting. 2021-08-11T16:05:45.592Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-08-11T16:05:45.610Z [qtp2045766957-18] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:45.609894Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-08-11T16:05:45.610Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:45.609894Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-08-11T16:05:45.611Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userworkflow HTTP/1.1 SC="500" 2021-08-11T16:05:45.632Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=8d9e1567-0e38-4414-b332-fe4e5068a185 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/consumers ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:45.631715Z InvocationID=e8ec8e18-fe3e-4e52-816b-c4172c24ce7f LogTimestamp=2021-08-11T16:05:45.631810Z Entering 2021-08-11T16:05:45.632Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=8d9e1567-0e38-4414-b332-fe4e5068a185 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/consumers ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:45.631715Z InvocationID=e8ec8e18-fe3e-4e52-816b-c4172c24ce7f LogTimestamp=2021-08-11T16:05:45.631810Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-08-11T16:05:45.676Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers EntryTimestamp=2021-08-11T16:05:45.632Z InvocationID=9d60f1c2-a5a8-4db6-866a-eaa6aca7fc4e PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id workflow 2021-08-11T16:05:46.211Z [qtp2045766957-14] ERROR o.o.s.b.m.o.impl.ConsumerOperation RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:45.632Z InvocationID=9d60f1c2-a5a8-4db6-866a-eaa6aca7fc4e ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null Error returned after get Consumer Data node workflow. status returned is NOT_FOUND 2021-08-11T16:05:46.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40454463-2bc2-44d9-bf95-329cb5df9ae5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:05:46.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40454463-2bc2-44d9-bf95-329cb5df9ae5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@62562555 2021-08-11T16:05:46.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40454463-2bc2-44d9-bf95-329cb5df9ae5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:05:46.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40454463-2bc2-44d9-bf95-329cb5df9ae5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:05:46.787Z [qtp2045766957-14] WARN o.o.s.b.a.impl.AuditingManager AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "workflow,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:45.632Z InvocationID=9d60f1c2-a5a8-4db6-866a-eaa6aca7fc4e ErrorCode=500 Failed to persist to cassandra auditing event: GENERAL_ERROR 2021-08-11T16:05:47.438Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "workflow,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=1805 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:45.632Z InvocationID=9d60f1c2-a5a8-4db6-866a-eaa6aca7fc4e ErrorCode=500 LogTimestamp=2021-08-11T16:05:47.437496Z ResponseDesc=Created Exiting. 2021-08-11T16:05:47.440Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-08-11T16:05:47.482Z [qtp2045766957-14] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:47.482097Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-08-11T16:05:47.482Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:47.482097Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-08-11T16:05:47.483Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userpolicy HTTP/1.1 SC="500" 2021-08-11T16:05:47.495Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=c4ca39c7-7001-4406-a467-5088f9f57c37 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/consumers ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:47.495412Z InvocationID=a677f48e-d9af-481c-9b8c-4a31c5ec9a19 LogTimestamp=2021-08-11T16:05:47.495484Z Entering 2021-08-11T16:05:47.496Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=c4ca39c7-7001-4406-a467-5088f9f57c37 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/consumers ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:47.495412Z InvocationID=a677f48e-d9af-481c-9b8c-4a31c5ec9a19 LogTimestamp=2021-08-11T16:05:47.495484Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-08-11T16:05:47.511Z [qtp2045766957-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers EntryTimestamp=2021-08-11T16:05:47.496Z InvocationID=fa37288a-04be-41e1-b304-a91b3910c136 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id policy 2021-08-11T16:05:48.071Z [qtp2045766957-15] ERROR o.o.s.b.m.o.impl.ConsumerOperation RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:47.496Z InvocationID=fa37288a-04be-41e1-b304-a91b3910c136 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null Error returned after get Consumer Data node policy. status returned is NOT_FOUND 2021-08-11T16:05:48.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:05:48.327Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:05:48.646Z [qtp2045766957-15] WARN o.o.s.b.a.impl.AuditingManager AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "policy,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:47.496Z InvocationID=fa37288a-04be-41e1-b304-a91b3910c136 ErrorCode=500 Failed to persist to cassandra auditing event: GENERAL_ERROR 2021-08-11T16:05:49.262Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "policy,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=1766 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:47.496Z InvocationID=fa37288a-04be-41e1-b304-a91b3910c136 ErrorCode=500 LogTimestamp=2021-08-11T16:05:49.262136Z ResponseDesc=Created Exiting. 2021-08-11T16:05:49.263Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-08-11T16:05:49.290Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:05:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:05:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:05:49.360Z [qtp2045766957-20] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:49.360427Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-08-11T16:05:49.360Z [qtp2045766957-20] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:49.360427Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-08-11T16:05:49.361Z [qtp2045766957-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/userpomba HTTP/1.1 SC="500" 2021-08-11T16:05:49.448Z [qtp2045766957-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=ee6f6769-4b13-4d6e-93de-eba54fb96d8d InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/consumers ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:49.448401Z InvocationID=0fbed851-b9dd-47f8-8c18-2cba8340e774 LogTimestamp=2021-08-11T16:05:49.448470Z Entering 2021-08-11T16:05:49.449Z [qtp2045766957-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=ee6f6769-4b13-4d6e-93de-eba54fb96d8d InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/consumers ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:49.448401Z InvocationID=0fbed851-b9dd-47f8-8c18-2cba8340e774 LogTimestamp=2021-08-11T16:05:49.448470Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-08-11T16:05:49.459Z [qtp2045766957-20] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers EntryTimestamp=2021-08-11T16:05:49.450Z InvocationID=58ae173d-9fb0-4946-9114-606d772b2bc4 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id pomba 2021-08-11T16:05:50.066Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef553b5b-13aa-4f26-86f7-cd825c0f7b65 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:05:50.066Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef553b5b-13aa-4f26-86f7-cd825c0f7b65 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@280b0d6e 2021-08-11T16:05:50.066Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef553b5b-13aa-4f26-86f7-cd825c0f7b65 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:05:50.066Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef553b5b-13aa-4f26-86f7-cd825c0f7b65 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:05:50.152Z [qtp2045766957-20] ERROR o.o.s.b.m.o.impl.ConsumerOperation RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:49.450Z InvocationID=58ae173d-9fb0-4946-9114-606d772b2bc4 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null Error returned after get Consumer Data node pomba. status returned is NOT_FOUND 2021-08-11T16:05:50.711Z [qtp2045766957-20] WARN o.o.s.b.a.impl.AuditingManager AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "pomba,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:49.450Z InvocationID=58ae173d-9fb0-4946-9114-606d772b2bc4 ErrorCode=500 Failed to persist to cassandra auditing event: GENERAL_ERROR 2021-08-11T16:05:51.574Z [qtp2045766957-20] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "pomba,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=2124 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:49.450Z InvocationID=58ae173d-9fb0-4946-9114-606d772b2bc4 ErrorCode=500 LogTimestamp=2021-08-11T16:05:51.574543Z ResponseDesc=Created Exiting. 2021-08-11T16:05:51.575Z [qtp2045766957-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-08-11T16:05:51.670Z [qtp2045766957-15] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:51.670411Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-08-11T16:05:51.670Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:51.670411Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-08-11T16:05:51.675Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermulticloud HTTP/1.1 SC="500" 2021-08-11T16:05:51.756Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=10a27f8b-e52b-46c9-86aa-a0e65f70c8e8 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/consumers ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:51.754520Z InvocationID=1535b4bf-ac05-4690-8b12-416f3f586432 LogTimestamp=2021-08-11T16:05:51.754600Z Entering 2021-08-11T16:05:51.756Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=10a27f8b-e52b-46c9-86aa-a0e65f70c8e8 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/consumers ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:51.754520Z InvocationID=1535b4bf-ac05-4690-8b12-416f3f586432 LogTimestamp=2021-08-11T16:05:51.754600Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-08-11T16:05:51.780Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers EntryTimestamp=2021-08-11T16:05:51.756Z InvocationID=990cdd6b-5d18-450e-b5e8-f50132f7ceed PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id multicloud 2021-08-11T16:05:52.565Z [qtp2045766957-14] ERROR o.o.s.b.m.o.impl.ConsumerOperation RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:51.756Z InvocationID=990cdd6b-5d18-450e-b5e8-f50132f7ceed ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null Error returned after get Consumer Data node multicloud. status returned is NOT_FOUND 2021-08-11T16:05:53.271Z [qtp2045766957-14] WARN o.o.s.b.a.impl.AuditingManager AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "multicloud,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:51.756Z InvocationID=990cdd6b-5d18-450e-b5e8-f50132f7ceed ErrorCode=500 Failed to persist to cassandra auditing event: GENERAL_ERROR 2021-08-11T16:05:53.273Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9838b33c-50bc-4c52-8890-34b5f3514988 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:05:53.273Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9838b33c-50bc-4c52-8890-34b5f3514988 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@65bc8533 2021-08-11T16:05:53.273Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9838b33c-50bc-4c52-8890-34b5f3514988 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:05:53.273Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9838b33c-50bc-4c52-8890-34b5f3514988 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:05:54.167Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "multicloud,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=2411 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:51.756Z InvocationID=990cdd6b-5d18-450e-b5e8-f50132f7ceed ErrorCode=500 LogTimestamp=2021-08-11T16:05:54.167579Z ResponseDesc=Created Exiting. 2021-08-11T16:05:54.168Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-08-11T16:05:54.198Z [qtp2045766957-12] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:54.198017Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-08-11T16:05:54.198Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:54.198017Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-08-11T16:05:54.198Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usercds HTTP/1.1 SC="500" 2021-08-11T16:05:54.276Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=8252a7ba-d653-44da-a787-83964e7c7eaa InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/consumers ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:54.276359Z InvocationID=86121731-d05d-4fd3-901c-b4170441d317 LogTimestamp=2021-08-11T16:05:54.276430Z Entering 2021-08-11T16:05:54.276Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=8252a7ba-d653-44da-a787-83964e7c7eaa InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/consumers ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:54.276359Z InvocationID=86121731-d05d-4fd3-901c-b4170441d317 LogTimestamp=2021-08-11T16:05:54.276430Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-08-11T16:05:54.304Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers EntryTimestamp=2021-08-11T16:05:54.277Z InvocationID=c14d4836-f28d-406a-8140-5e8783f067aa PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id cds 2021-08-11T16:05:54.887Z [qtp2045766957-18] ERROR o.o.s.b.m.o.impl.ConsumerOperation RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:54.277Z InvocationID=c14d4836-f28d-406a-8140-5e8783f067aa ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null Error returned after get Consumer Data node cds. status returned is NOT_FOUND 2021-08-11T16:05:55.475Z [qtp2045766957-18] WARN o.o.s.b.a.impl.AuditingManager AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "cds,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:54.277Z InvocationID=c14d4836-f28d-406a-8140-5e8783f067aa ErrorCode=500 Failed to persist to cassandra auditing event: GENERAL_ERROR 2021-08-11T16:05:55.967Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=993f37c2-d327-47c5-a5b5-bdf1fd1a1298 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:05:55.967Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=993f37c2-d327-47c5-a5b5-bdf1fd1a1298 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7f312fcc 2021-08-11T16:05:55.967Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=993f37c2-d327-47c5-a5b5-bdf1fd1a1298 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:05:55.967Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=993f37c2-d327-47c5-a5b5-bdf1fd1a1298 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:05:56.172Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "cds,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=1895 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:54.277Z InvocationID=c14d4836-f28d-406a-8140-5e8783f067aa ErrorCode=500 LogTimestamp=2021-08-11T16:05:56.172714Z ResponseDesc=Created Exiting. 2021-08-11T16:05:56.173Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-08-11T16:05:56.183Z [qtp2045766957-17] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:56.183829Z StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-08-11T16:05:56.183Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:05:56.183829Z StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-08-11T16:05:56.184Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/usermodeling HTTP/1.1 SC="500" 2021-08-11T16:05:56.285Z [qtp2045766957-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=a83e259d-651e-4b62-9a3f-c7094013dc7f InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/consumers ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:56.266305Z InvocationID=a3a5f829-3e89-4f6e-88ed-92031a0ae7f3 LogTimestamp=2021-08-11T16:05:56.266399Z Entering 2021-08-11T16:05:56.286Z [qtp2045766957-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=a83e259d-651e-4b62-9a3f-c7094013dc7f InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/consumers ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:56.266305Z InvocationID=a3a5f829-3e89-4f6e-88ed-92031a0ae7f3 LogTimestamp=2021-08-11T16:05:56.266399Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/consumers HTTP/1.1 2021-08-11T16:05:56.365Z [qtp2045766957-11] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers EntryTimestamp=2021-08-11T16:05:56.286Z InvocationID=2399e6b1-c0d8-4c8d-842d-153affb4d27a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id modeling 2021-08-11T16:05:56.896Z [qtp2045766957-11] ERROR o.o.s.b.m.o.impl.ConsumerOperation RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:56.286Z InvocationID=2399e6b1-c0d8-4c8d-842d-153affb4d27a ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null Error returned after get Consumer Data node modeling. status returned is NOT_FOUND 2021-08-11T16:05:57.412Z [qtp2045766957-11] WARN o.o.s.b.a.impl.AuditingManager AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "modeling,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:56.286Z InvocationID=2399e6b1-c0d8-4c8d-842d-153affb4d27a ErrorCode=500 Failed to persist to cassandra auditing event: GENERAL_ERROR 2021-08-11T16:05:57.951Z [qtp2045766957-11] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "AddECOMPUserCredentials" MODIFIER = "Jimmy Hendrix(jh0003)" ECOMP_USER = "modeling,9cd4c3ad2a6f6ce3f3414e68b5157e63,35371c046f88c603ccba152cb3db34ec4475cb2e5713f2fc0a43bf18a5243495" STATUS = "201" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=1665 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/consumers TargetServiceName=null EntryTimestamp=2021-08-11T16:05:56.286Z InvocationID=2399e6b1-c0d8-4c8d-842d-153affb4d27a ErrorCode=500 LogTimestamp=2021-08-11T16:05:57.951261Z ResponseDesc=Created Exiting. 2021-08-11T16:05:57.951Z [qtp2045766957-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/consumers HTTP/1.1 SC="201" 2021-08-11T16:05:58.153Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=7e77b6b7-9057-407e-a35e-0059362bc443 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:58.153330Z InvocationID=c6c5c6bb-ef1d-4cdc-ba1a-89d2e76c9ac1 LogTimestamp=2021-08-11T16:05:58.153484Z Entering 2021-08-11T16:05:58.154Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=7e77b6b7-9057-407e-a35e-0059362bc443 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/user/jh0003 ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:58.153330Z InvocationID=c6c5c6bb-ef1d-4cdc-ba1a-89d2e76c9ac1 LogTimestamp=2021-08-11T16:05:58.153484Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 2021-08-11T16:05:58.169Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=15 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/user/jh0003 EntryTimestamp=2021-08-11T16:05:58.154Z InvocationID=7faabfe7-3563-49b7-a8c4-90dcb4605c41 LogTimestamp=2021-08-11T16:05:58.169554Z ResponseDesc=OK Exiting. 2021-08-11T16:05:58.170Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/user/jh0003 HTTP/1.1 SC="200" 2021-08-11T16:05:58.540Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=b56747e4-39cd-4840-a310-9f91205748d4 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/screen ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:58.539900Z InvocationID=8ef2d20f-31ac-4a21-8666-7c63b89bbb36 LogTimestamp=2021-08-11T16:05:58.539959Z Entering 2021-08-11T16:05:58.540Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=b56747e4-39cd-4840-a310-9f91205748d4 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/screen ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:58.539900Z InvocationID=8ef2d20f-31ac-4a21-8666-7c63b89bbb36 LogTimestamp=2021-08-11T16:05:58.539959Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 GET /sdc2/rest/v1/screen HTTP/1.1 2021-08-11T16:05:58.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=070a0ae9-73fe-424b-ae39-d46b7493c61c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:05:58.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=070a0ae9-73fe-424b-ae39-d46b7493c61c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2c9ed1f2 2021-08-11T16:05:58.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=070a0ae9-73fe-424b-ae39-d46b7493c61c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:05:58.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=070a0ae9-73fe-424b-ae39-d46b7493c61c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:05:58.681Z [qtp2045766957-15] INFO o.o.s.b.m.j.o.ToscaElementOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/screen EntryTimestamp=2021-08-11T16:05:58.540Z InvocationID=6446b474-d37c-45f2-8e79-33ed5b33154c PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p time to fetch all catalog elements: StopWatch '': running time = 30859741 ns\n---------------------------------------------\nns % Task name\n---------------------------------------------\n030859741 100% \n 2021-08-11T16:05:58.703Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=162 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/screen EntryTimestamp=2021-08-11T16:05:58.540Z InvocationID=6446b474-d37c-45f2-8e79-33ed5b33154c LogTimestamp=2021-08-11T16:05:58.702643Z ResponseDesc=OK Exiting. 2021-08-11T16:05:58.703Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc2/rest/v1/screen HTTP/1.1 SC="200" 2021-08-11T16:05:58.728Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=dfcc13ed-0eab-463d-b33a-5265d7b7454b InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:58.728208Z InvocationID=f3cb5cd4-3188-4834-8d3f-9a90476b34f1 LogTimestamp=2021-08-11T16:05:58.728277Z Entering 2021-08-11T16:05:58.728Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=dfcc13ed-0eab-463d-b33a-5265d7b7454b InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/uploadType/datatypes ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:05:58.728208Z InvocationID=f3cb5cd4-3188-4834-8d3f-9a90476b34f1 LogTimestamp=2021-08-11T16:05:58.728277Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 2021-08-11T16:05:59.144Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.Root to dao for create 2021-08-11T16:05:59.238Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : integer to dao for create 2021-08-11T16:06:00.357Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : string to dao for create 2021-08-11T16:06:00.386Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : boolean to dao for create 2021-08-11T16:06:00.407Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : float to dao for create 2021-08-11T16:06:00.431Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : list to dao for create 2021-08-11T16:06:00.451Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : map to dao for create 2021-08-11T16:06:00.470Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : json to dao for create 2021-08-11T16:06:00.479Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : scalar-unit to dao for create 2021-08-11T16:06:00.541Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : scalar-unit.size to dao for create 2021-08-11T16:06:00.573Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : scalar-unit.time to dao for create 2021-08-11T16:06:00.607Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : scalar-unit.frequency to dao for create 2021-08-11T16:06:00.619Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.Credential to dao for create 2021-08-11T16:06:00.894Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.PortMirroringConnectionPointDescription to dao for create 2021-08-11T16:06:00.949Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.TimeInterval to dao for create 2021-08-11T16:06:00.970Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.network.NetworkInfo to dao for create 2021-08-11T16:06:01.023Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.network.PortInfo to dao for create 2021-08-11T16:06:01.104Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.network.PortDef to dao for create 2021-08-11T16:06:01.120Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.network.PortSpec to dao for create 2021-08-11T16:06:01.223Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.heat.network.AddressPair to dao for create 2021-08-11T16:06:01.244Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.heat.network.subnet.HostRoute to dao for create 2021-08-11T16:06:01.263Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.heat.network.AllocationPool to dao for create 2021-08-11T16:06:01.292Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.heat.network.neutron.Subnet to dao for create 2021-08-11T16:06:01.601Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.heat.novaServer.network.PortExtraProperties to dao for create 2021-08-11T16:06:01.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0741c6ab-099b-4cca-8570-6df808b05192 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:06:01.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0741c6ab-099b-4cca-8570-6df808b05192 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5c0ed972 2021-08-11T16:06:01.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0741c6ab-099b-4cca-8570-6df808b05192 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:06:01.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0741c6ab-099b-4cca-8570-6df808b05192 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:06:01.704Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.heat.novaServer.network.AddressInfo to dao for create 2021-08-11T16:06:01.778Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.heat.neutron.port.FixedIps to dao for create 2021-08-11T16:06:01.830Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.heat.FileInfo to dao for create 2021-08-11T16:06:01.854Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.heat.contrail.network.rule.PortPairs to dao for create 2021-08-11T16:06:01.875Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.heat.contrail.network.rule.Rule to dao for create 2021-08-11T16:06:01.929Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.heat.contrail.network.rule.RuleList to dao for create 2021-08-11T16:06:01.949Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.heat.contrail.network.rule.VirtualNetwork to dao for create 2021-08-11T16:06:01.964Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.heat.network.neutron.SecurityRules.Rule to dao for create 2021-08-11T16:06:02.046Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.heat.substitution.SubstitutionFiltering to dao for create 2021-08-11T16:06:02.090Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.virtual.network.rule.RefDataSequence to dao for create 2021-08-11T16:06:02.105Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.virtual.network.rule.RefData to dao for create 2021-08-11T16:06:02.119Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.virtual.network.rule.ref.data.IpamSubnet to dao for create 2021-08-11T16:06:02.135Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.virtual.network.rule.ref.data.IpamSubnetList to dao for create 2021-08-11T16:06:02.153Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.virtual.network.rule.IpamRefData to dao for create 2021-08-11T16:06:02.164Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.network.rule.SrcVirtualNetwork to dao for create 2021-08-11T16:06:02.182Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.network.rule.DstVirtualNetwork to dao for create 2021-08-11T16:06:02.192Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.network.rule.DstPortPairs to dao for create 2021-08-11T16:06:02.214Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.network.rule.SrcPortPairs to dao for create 2021-08-11T16:06:02.235Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.network.rule.ActionList to dao for create 2021-08-11T16:06:02.269Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.network.rule.Rule to dao for create 2021-08-11T16:06:02.323Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.network.rule.RuleList to dao for create 2021-08-11T16:06:02.334Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.heat.network.contrail.port.StaticRoute to dao for create 2021-08-11T16:06:02.355Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.heat.network.contrail.AddressPair to dao for create 2021-08-11T16:06:02.383Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.heat.network.contrail.InterfaceData to dao for create 2021-08-11T16:06:02.410Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.virtual.machine.interface.Properties to dao for create 2021-08-11T16:06:02.422Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.Root to dao for create 2021-08-11T16:06:02.437Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.AssignmentRequirements to dao for create 2021-08-11T16:06:02.451Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.network.SubnetAssignments to dao for create 2021-08-11T16:06:02.494Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.network.IPv4SubnetAssignments to dao for create 2021-08-11T16:06:02.510Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.network.IPv6SubnetAssignments to dao for create 2021-08-11T16:06:02.530Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.network.NetworkAssignments to dao for create 2021-08-11T16:06:02.602Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.network.ProviderNetwork to dao for create 2021-08-11T16:06:02.644Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.network.NetworkFlows to dao for create 2021-08-11T16:06:02.674Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.network.VlanRequirements to dao for create 2021-08-11T16:06:02.697Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.network.IpRequirements to dao for create 2021-08-11T16:06:02.780Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.network.MacAssignments to dao for create 2021-08-11T16:06:02.803Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.CSProperties to dao for create 2021-08-11T16:06:02.909Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.NSCapabilities to dao for create 2021-08-11T16:06:03.047Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.NSSCapabilities to dao for create 2021-08-11T16:06:03.179Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.ServiceProfile to dao for create 2021-08-11T16:06:03.288Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.SliceProfile to dao for create 2021-08-11T16:06:03.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:06:03.332Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:06:03.430Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.EcompHoming to dao for create 2021-08-11T16:06:03.453Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.EcompNaming to dao for create 2021-08-11T16:06:03.485Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.network.MacRequirements to dao for create 2021-08-11T16:06:03.515Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.virtual.machine.subInterface.AddressPairIp to dao for create 2021-08-11T16:06:03.528Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.virtual.machine.subInterface.MacAddress to dao for create 2021-08-11T16:06:03.543Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.virtual.machine.subInterface.Properties to dao for create 2021-08-11T16:06:03.554Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.virtual.machine.subInterface.AddressPair to dao for create 2021-08-11T16:06:03.580Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.heat.contrailV2.virtual.machine.subInterface.AddressPairs to dao for create 2021-08-11T16:06:03.607Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.Naming to dao for create 2021-08-11T16:06:03.632Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.flavors.ComputeFlavor to dao for create 2021-08-11T16:06:03.660Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.flavors.LicenseFlavor to dao for create 2021-08-11T16:06:03.676Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.flavors.VendorInfo to dao for create 2021-08-11T16:06:03.693Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.flavors.DeploymentFlavor to dao for create 2021-08-11T16:06:03.724Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.ImageInfo to dao for create 2021-08-11T16:06:03.751Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : org.openecomp.datatypes.network.RelatedNetworksAssignments to dao for create 2021-08-11T16:06:03.763Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.VnfcAdditionalConfigurableProperties to dao for create 2021-08-11T16:06:03.776Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.VnfcConfigurableProperties to dao for create 2021-08-11T16:06:03.790Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.RequestedAdditionalCapability to dao for create 2021-08-11T16:06:03.823Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.VirtualMemory to dao for create 2021-08-11T16:06:03.843Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.VirtualCpuPinning to dao for create 2021-08-11T16:06:03.860Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.VirtualCpu to dao for create 2021-08-11T16:06:03.890Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.LogicalNodeData to dao for create 2021-08-11T16:06:03.899Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.VirtualNetworkInterfaceRequirements to dao for create 2021-08-11T16:06:03.925Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.injectFile to dao for create 2021-08-11T16:06:03.935Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:06:03.936Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.L2AddressData to dao for create 2021-08-11T16:06:03.943Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.L3AddressData to dao for create 2021-08-11T16:06:03.966Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.AddressData to dao for create 2021-08-11T16:06:04.007Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.ConnectivityType to dao for create 2021-08-11T16:06:04.025Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.VlFlavour to dao for create 2021-08-11T16:06:04.040Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.ext.AddressPairs to dao for create 2021-08-11T16:06:04.057Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.ext.FloatingIP to dao for create 2021-08-11T16:06:04.076Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.ext.LocationInfo to dao for create 2021-08-11T16:06:04.095Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.ext.HostRouteInfo to dao for create 2021-08-11T16:06:04.110Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.ext.InjectData to dao for create 2021-08-11T16:06:04.145Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.ext.zte.WatchDog to dao for create 2021-08-11T16:06:04.169Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.ext.LocalAffinityOrAntiAffinityRule to dao for create 2021-08-11T16:06:04.189Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.VduProfile to dao for create 2021-08-11T16:06:04.213Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.LinkBitrateRequirements to dao for create 2021-08-11T16:06:04.256Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.Qos to dao for create 2021-08-11T16:06:04.311Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.IpAllocationPool to dao for create 2021-08-11T16:06:04.331Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.L2ProtocolData to dao for create 2021-08-11T16:06:04.381Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.L3ProtocolData to dao for create 2021-08-11T16:06:04.414Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.VirtualLinkProtocolData to dao for create 2021-08-11T16:06:04.440Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.VlProfile to dao for create 2021-08-11T16:06:04.492Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.VduLevel to dao for create 2021-08-11T16:06:04.501Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.ScaleInfo to dao for create 2021-08-11T16:06:04.519Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.ScalingAspect to dao for create 2021-08-11T16:06:04.568Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.InstantiationLevel to dao for create 2021-08-11T16:06:04.579Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.VnfInstantiateOperationConfiguration to dao for create 2021-08-11T16:06:04.587Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.VnfScaleOperationConfiguration to dao for create 2021-08-11T16:06:04.594Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.VnfScaleToLevelOperationConfiguration to dao for create 2021-08-11T16:06:04.653Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.VnfHealOperationConfiguration to dao for create 2021-08-11T16:06:04.674Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.VnfTerminateOperationConfiguration to dao for create 2021-08-11T16:06:04.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a8b47223-af31-42f0-a7f8-6bac0a50e84b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:06:04.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a8b47223-af31-42f0-a7f8-6bac0a50e84b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5d1a5add 2021-08-11T16:06:04.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a8b47223-af31-42f0-a7f8-6bac0a50e84b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:06:04.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a8b47223-af31-42f0-a7f8-6bac0a50e84b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:06:04.692Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.VnfOperateOperationConfiguration to dao for create 2021-08-11T16:06:04.808Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.VnfChangeFlavourOperationConfiguration to dao for create 2021-08-11T16:06:04.829Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.VnfProfile to dao for create 2021-08-11T16:06:04.893Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.UriAuthority to dao for create 2021-08-11T16:06:04.926Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.UriComponents to dao for create 2021-08-11T16:06:04.967Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.InterfaceDetails to dao for create 2021-08-11T16:06:04.990Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.VirtualLinkMonitoringParameter to dao for create 2021-08-11T16:06:05.064Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.VnfcMonitoringParameter to dao for create 2021-08-11T16:06:05.151Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.VnfChangeExtConnectivityOperationConfiguration to dao for create 2021-08-11T16:06:05.162Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send DATA_TYPE : tosca.datatypes.nfv.VnfMonitoringParameter to dao for create 2021-08-11T16:06:05.194Z [qtp2045766957-14] WARN o.o.s.b.c.impl.DataTypeImportManager RequestID=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null Creating data type tosca.datatypes.nfv.VnfChangeCurrentPackageOperationConfiguration which derived from abstract data type with no properties 2021-08-11T16:06:05.195Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VnfChangeCurrentPackageOperationConfiguration to dao for create 2021-08-11T16:06:05.200Z [qtp2045766957-14] WARN o.o.s.b.c.impl.DataTypeImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 Creating data type tosca.datatypes.nfv.VnfCreateSnapshotOperationConfiguration which derived from abstract data type with no properties 2021-08-11T16:06:05.201Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VnfCreateSnapshotOperationConfiguration to dao for create 2021-08-11T16:06:05.207Z [qtp2045766957-14] WARN o.o.s.b.c.impl.DataTypeImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 Creating data type tosca.datatypes.nfv.VnfRevertToSnapshotOperationConfiguration which derived from abstract data type with no properties 2021-08-11T16:06:05.208Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VnfRevertToSnapshotOperationConfiguration to dao for create 2021-08-11T16:06:05.249Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VnfLcmOperationsConfiguration to dao for create 2021-08-11T16:06:05.356Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.CpProtocolData to dao for create 2021-08-11T16:06:05.374Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VnfAdditionalConfigurableProperties to dao for create 2021-08-11T16:06:05.393Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VnfmInterfaceInfo to dao for create 2021-08-11T16:06:05.419Z [qtp2045766957-14] WARN o.o.s.b.c.impl.DataTypeImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 Creating data type tosca.datatypes.nfv.OauthServerInfo which derived from abstract data type with no properties 2021-08-11T16:06:05.420Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.OauthServerInfo to dao for create 2021-08-11T16:06:05.425Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VnfConfigurableProperties to dao for create 2021-08-11T16:06:05.466Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VnfInfoModifiableAttributesExtensions to dao for create 2021-08-11T16:06:05.477Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VnfInfoModifiableAttributesMetadata to dao for create 2021-08-11T16:06:05.487Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VnfInfoModifiableAttributes to dao for create 2021-08-11T16:06:05.513Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VirtualLinkBitrateLevel to dao for create 2021-08-11T16:06:05.555Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VnfOperationAdditionalParameters to dao for create 2021-08-11T16:06:05.563Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VirtualFileStorageData to dao for create 2021-08-11T16:06:05.584Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VirtualObjectStorageData to dao for create 2021-08-11T16:06:05.593Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.VirtualBlockStorageData to dao for create 2021-08-11T16:06:05.614Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.ChecksumData to dao for create 2021-08-11T16:06:05.632Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.SwImageData to dao for create 2021-08-11T16:06:05.705Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.NsVirtualLinkQos to dao for create 2021-08-11T16:06:05.737Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.ServiceAvailability to dao for create 2021-08-11T16:06:05.756Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.NsVlProfile to dao for create 2021-08-11T16:06:05.830Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.NsProfile to dao for create 2021-08-11T16:06:05.851Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.MciopProfile to dao for create 2021-08-11T16:06:05.911Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.BootDataVimSpecificProperties to dao for create 2021-08-11T16:06:05.952Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.KvpData to dao for create 2021-08-11T16:06:05.962Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.ContentOrFileData to dao for create 2021-08-11T16:06:05.999Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.BootData to dao for create 2021-08-11T16:06:06.028Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.ServicePortData to dao for create 2021-08-11T16:06:06.051Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.AdditionalServiceData to dao for create 2021-08-11T16:06:06.076Z [qtp2045766957-14] WARN o.o.s.b.c.impl.DataTypeImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 Creating data type tosca.datatypes.nfv.NsOperationAdditionalParameters which derived from abstract data type with no properties 2021-08-11T16:06:06.077Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.NsOperationAdditionalParameters to dao for create 2021-08-11T16:06:06.083Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.Mask to dao for create 2021-08-11T16:06:06.106Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : tosca.datatypes.nfv.NsMonitoringParameter to dao for create 2021-08-11T16:06:06.126Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : onap.datatypes.partner.access_details to dao for create 2021-08-11T16:06:06.169Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : onap.datatypes.partner.bandwidth to dao for create 2021-08-11T16:06:06.194Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : onap.datatypes.partner.service_details to dao for create 2021-08-11T16:06:06.247Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 send DATA_TYPE : onap.datatypes.partner.sppartner_details to dao for create 2021-08-11T16:06:06.265Z [qtp2045766957-14] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 all DataTypeDefinition(derivedFrom=null, constraints=null, properties=[PropertyDefinition [ PropertyDataDefinition(definition=false, hidden=false, uniqueId=onap.datatypes.partner.sppartner_details.datatype.partner_url, type=string, required=false, defaultValue=null, description=Partner URL to access and provision the service, schema=null, password=false, name=partner_url, value=null, label=null, immutable=false, mappedToComponentProperty=true, toscaGetFunctionType=null, inputPath=null, status=null, inputId=null, instanceUniqueId=null, propertyId=null, parentPropertyType=null, subPropertyInputPath=null, annotations=null, parentUniqueId=null, getInputValues=null, isDeclaredListInput=false, getPolicyValues=null, propertyConstraints=null, metadata=null), name=partner_url, constraints=null]], PropertyDefinition [ PropertyDataDefinition(definition=false, hidden=false, uniqueId=onap.datatypes.partner.sppartner_details.datatype.auth_user, type=string, required=false, defaultValue=null, description=Authentication details to access the Service URL, schema=null, password=false, name=auth_user, value=null, label=null, immutable=false, mappedToComponentProperty=true, toscaGetFunctionType=null, inputPath=null, status=null, inputId=null, instanceUniqueId=null, propertyId=null, parentPropertyType=null, subPropertyInputPath=null, annotations=null, parentUniqueId=null, getInputValues=null, isDeclaredListInput=false, getPolicyValues=null, propertyConstraints=null, metadata=null), name=auth_user, constraints=null]], PropertyDefinition [ PropertyDataDefinition(definition=false, hidden=false, uniqueId=onap.datatypes.partner.sppartner_details.datatype.auth_password, type=string, required=false, defaultValue=null, description=Authentication details to access the Service URL, schema=null, password=false, name=auth_password, value=null, label=null, immutable=false, mappedToComponentProperty=true, toscaGetFunctionType=null, inputPath=null, status=null, inputId=null, instanceUniqueId=null, propertyId=null, parentPropertyType=null, subPropertyInputPath=null, annotations=null, parentUniqueId=null, getInputValues=null, isDeclaredListInput=false, getPolicyValues=null, propertyConstraints=null, metadata=null), name=auth_password, constraints=null]], PropertyDefinition [ PropertyDataDefinition(definition=false, hidden=false, uniqueId=onap.datatypes.partner.sppartner_details.datatype.uuid, type=string, required=false, defaultValue=null, description=UUID of the service definition defined in the partner catalog, schema=null, password=false, name=uuid, value=null, label=null, immutable=false, mappedToComponentProperty=true, toscaGetFunctionType=null, inputPath=null, status=null, inputId=null, instanceUniqueId=null, propertyId=null, parentPropertyType=null, subPropertyInputPath=null, annotations=null, parentUniqueId=null, getInputValues=null, isDeclaredListInput=false, getPolicyValues=null, propertyConstraints=null, metadata=null), name=uuid, constraints=null]]]) were created successfully!!! 2021-08-11T16:06:07.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01e0245f-b3b7-4272-a1a3-dc6e911caaf9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:06:07.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01e0245f-b3b7-4272-a1a3-dc6e911caaf9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2ba341fd 2021-08-11T16:06:07.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01e0245f-b3b7-4272-a1a3-dc6e911caaf9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:06:07.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01e0245f-b3b7-4272-a1a3-dc6e911caaf9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:06:10.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b15777e3-fc14-4aee-8c79-41ce7f7bed68 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:06:10.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b15777e3-fc14-4aee-8c79-41ce7f7bed68 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@71e8143a 2021-08-11T16:06:10.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b15777e3-fc14-4aee-8c79-41ce7f7bed68 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:06:10.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b15777e3-fc14-4aee-8c79-41ce7f7bed68 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:06:13.202Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=2006c540-34d5-47e5-b18a-b74a9d752de4 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=dcae ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.24.206 StatusCode=INPROGRESS RemoteHost=10.242.24.206 ServiceName=v1/artifactTypes ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:06:13.201352Z InvocationID=efbf3c8f-aa1a-4091-b4f0-91f10d0d0d4b LogTimestamp=2021-08-11T16:06:13.201692Z Entering 2021-08-11T16:06:13.202Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=2006c540-34d5-47e5-b18a-b74a9d752de4 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=dcae ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.24.206 StatusCode=INPROGRESS userId=null RemoteHost=10.242.24.206 ServiceName=v1/artifactTypes ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:06:13.201352Z InvocationID=efbf3c8f-aa1a-4091-b4f0-91f10d0d0d4b LogTimestamp=2021-08-11T16:06:13.201692Z localAddr=10.242.24.223 remoteAddr=10.242.24.206 GET /sdc/v1/artifactTypes HTTP/1.1 2021-08-11T16:06:13.208Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=5 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/artifactTypes EntryTimestamp=2021-08-11T16:06:13.203Z InvocationID=2daa5e2d-268e-406a-8b4e-04aac269e790 LogTimestamp=2021-08-11T16:06:13.208252Z ResponseDesc=OK Exiting. 2021-08-11T16:06:13.208Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc/v1/artifactTypes HTTP/1.1 SC="200" 2021-08-11T16:06:13.546Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=52e8e6dc-fe55-4aee-874a-faaaf170d72b InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=dcae ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.24.206 StatusCode=INPROGRESS RemoteHost=10.242.24.206 ServiceName=v1/registerForDistribution ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:06:13.545703Z InvocationID=990679c0-4df2-475e-8cda-47d687ff3a72 LogTimestamp=2021-08-11T16:06:13.545771Z Entering 2021-08-11T16:06:13.546Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=52e8e6dc-fe55-4aee-874a-faaaf170d72b InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=dcae ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.24.206 StatusCode=INPROGRESS userId=null RemoteHost=10.242.24.206 ServiceName=v1/registerForDistribution ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:06:13.545703Z InvocationID=990679c0-4df2-475e-8cda-47d687ff3a72 LogTimestamp=2021-08-11T16:06:13.545771Z localAddr=10.242.24.223 remoteAddr=10.242.24.206 POST /sdc/v1/registerForDistribution HTTP/1.1 2021-08-11T16:06:13.552Z [qtp2045766957-18] WARN c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:06:13.546Z InvocationID=3111346d-1e89-47e5-8e3a-725590e1f34e PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/producers/JdgsoOp5Iu9y36oi will send credentials over a clear channel. 2021-08-11T16:06:13.552Z [qtp2045766957-18] INFO c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:06:13.546Z InvocationID=3111346d-1e89-47e5-8e3a-725590e1f34e PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/producers/JdgsoOp5Iu9y36oi (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:06:13.633Z [qtp2045766957-18] INFO c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:06:13.546Z InvocationID=3111346d-1e89-47e5-8e3a-725590e1f34e PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p \t--> HTTP/1.1 200 OK 2021-08-11T16:06:13.650Z [qtp2045766957-18] WARN c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "JdgsoOp5Iu9y36oi" STATUS = "200" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:06:13.546Z InvocationID=3111346d-1e89-47e5-8e3a-725590e1f34e PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/consumers/JdgsoOp5Iu9y36oi will send credentials over a clear channel. 2021-08-11T16:06:13.650Z [qtp2045766957-18] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "JdgsoOp5Iu9y36oi" STATUS = "200" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:06:13.546Z InvocationID=3111346d-1e89-47e5-8e3a-725590e1f34e PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/consumers/JdgsoOp5Iu9y36oi (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:06:13.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb349e40-b24c-42e7-8e7f-2b4b31f3e4d7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:06:13.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb349e40-b24c-42e7-8e7f-2b4b31f3e4d7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5b07963c 2021-08-11T16:06:13.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb349e40-b24c-42e7-8e7f-2b4b31f3e4d7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:06:13.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb349e40-b24c-42e7-8e7f-2b4b31f3e4d7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:06:13.723Z [qtp2045766957-18] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "JdgsoOp5Iu9y36oi" STATUS = "200" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:06:13.546Z InvocationID=3111346d-1e89-47e5-8e3a-725590e1f34e PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p \t--> HTTP/1.1 200 OK 2021-08-11T16:06:13.746Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "DRegister" CONSUMER_ID = "ves-openapi-manager" API_KEY = "JdgsoOp5Iu9y36oi" D_ENV = "AUTO" STATUS = "200" DESC = "" DNOTIF_TOPIC = "SDC-DISTR-NOTIF-TOPIC-AUTO" DSTATUS_TOPIC = "SDC-DISTR-STATUS-TOPIC-AUTO" ResponseCode=200 RequestID=null ElapsedTime=200 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:06:13.546Z InvocationID=3111346d-1e89-47e5-8e3a-725590e1f34e LogTimestamp=2021-08-11T16:06:13.746122Z ResponseDesc=OK Exiting. 2021-08-11T16:06:13.746Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc/v1/registerForDistribution HTTP/1.1 SC="200" 2021-08-11T16:06:14.427Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=7eb41be4-d1f3-40a4-9f95-974dedc2a377 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=mso auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.50.88 StatusCode=INPROGRESS RemoteHost=10.242.50.88 ServerIPAddress=10.242.24.223 ServiceName=v1/artifactTypes ServiceInstanceID=null EntryTimestamp=2021-08-11T16:06:14.427304Z InvocationID=90cadfab-fb2d-48a2-8201-1cb1340b2cce LogTimestamp=2021-08-11T16:06:14.427391Z Entering 2021-08-11T16:06:14.427Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=7eb41be4-d1f3-40a4-9f95-974dedc2a377 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=mso auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.50.88 StatusCode=INPROGRESS userId=null RemoteHost=10.242.50.88 ServerIPAddress=10.242.24.223 ServiceName=v1/artifactTypes ServiceInstanceID=null EntryTimestamp=2021-08-11T16:06:14.427304Z InvocationID=90cadfab-fb2d-48a2-8201-1cb1340b2cce LogTimestamp=2021-08-11T16:06:14.427391Z localAddr=10.242.24.223 remoteAddr=10.242.50.88 GET /sdc/v1/artifactTypes HTTP/1.1 2021-08-11T16:06:14.429Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=1 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.8) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/artifactTypes EntryTimestamp=2021-08-11T16:06:14.427Z InvocationID=07e4e856-c64f-4129-9c06-8b58c05f9cec LogTimestamp=2021-08-11T16:06:14.428901Z ResponseDesc=OK Exiting. 2021-08-11T16:06:14.429Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc/v1/artifactTypes HTTP/1.1 SC="200" 2021-08-11T16:06:14.655Z [qtp2045766957-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=36fc35a1-b8ea-4d43-b4ec-326533e72aa0 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=mso auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.50.88 StatusCode=INPROGRESS RemoteHost=10.242.50.88 ServerIPAddress=10.242.24.223 ServiceName=v1/registerForDistribution ServiceInstanceID=null EntryTimestamp=2021-08-11T16:06:14.655011Z InvocationID=4d8c6560-eaed-484d-b19c-de07309debdf LogTimestamp=2021-08-11T16:06:14.655107Z Entering 2021-08-11T16:06:14.655Z [qtp2045766957-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=36fc35a1-b8ea-4d43-b4ec-326533e72aa0 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=mso auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.50.88 StatusCode=INPROGRESS userId=null RemoteHost=10.242.50.88 ServerIPAddress=10.242.24.223 ServiceName=v1/registerForDistribution ServiceInstanceID=null EntryTimestamp=2021-08-11T16:06:14.655011Z InvocationID=4d8c6560-eaed-484d-b19c-de07309debdf LogTimestamp=2021-08-11T16:06:14.655107Z localAddr=10.242.24.223 remoteAddr=10.242.50.88 POST /sdc/v1/registerForDistribution HTTP/1.1 2021-08-11T16:06:14.658Z [qtp2045766957-11] WARN c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:06:14.655Z InvocationID=99f7bc2c-012d-4c36-94ae-4bb853f5177a PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.8) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/producers/plIJ4ti4apyYLAPB will send credentials over a clear channel. 2021-08-11T16:06:14.658Z [qtp2045766957-11] INFO c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:06:14.655Z InvocationID=99f7bc2c-012d-4c36-94ae-4bb853f5177a PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.8) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/producers/plIJ4ti4apyYLAPB (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:06:14.729Z [qtp2045766957-11] INFO c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:06:14.655Z InvocationID=99f7bc2c-012d-4c36-94ae-4bb853f5177a PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.8) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p \t--> HTTP/1.1 200 OK 2021-08-11T16:06:14.745Z [qtp2045766957-11] WARN c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "plIJ4ti4apyYLAPB" STATUS = "200" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:06:14.655Z InvocationID=99f7bc2c-012d-4c36-94ae-4bb853f5177a PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.8) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/consumers/plIJ4ti4apyYLAPB will send credentials over a clear channel. 2021-08-11T16:06:14.745Z [qtp2045766957-11] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "plIJ4ti4apyYLAPB" STATUS = "200" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:06:14.655Z InvocationID=99f7bc2c-012d-4c36-94ae-4bb853f5177a PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.8) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/consumers/plIJ4ti4apyYLAPB (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:06:14.807Z [qtp2045766957-11] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "plIJ4ti4apyYLAPB" STATUS = "200" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:06:14.655Z InvocationID=99f7bc2c-012d-4c36-94ae-4bb853f5177a PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.8) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p \t--> HTTP/1.1 200 OK 2021-08-11T16:06:14.818Z [qtp2045766957-11] WARN c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "CONSUMER" API_KEY = "plIJ4ti4apyYLAPB" STATUS = "200" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:06:14.655Z InvocationID=99f7bc2c-012d-4c36-94ae-4bb853f5177a PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.8) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/consumers/plIJ4ti4apyYLAPB will send credentials over a clear channel. 2021-08-11T16:06:14.818Z [qtp2045766957-11] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "CONSUMER" API_KEY = "plIJ4ti4apyYLAPB" STATUS = "200" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:06:14.655Z InvocationID=99f7bc2c-012d-4c36-94ae-4bb853f5177a PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.8) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/consumers/plIJ4ti4apyYLAPB (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:06:14.903Z [qtp2045766957-11] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "CONSUMER" API_KEY = "plIJ4ti4apyYLAPB" STATUS = "200" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:06:14.655Z InvocationID=99f7bc2c-012d-4c36-94ae-4bb853f5177a PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.8) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p \t--> HTTP/1.1 200 OK 2021-08-11T16:06:14.915Z [qtp2045766957-11] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "DRegister" CONSUMER_ID = "SO-COpenSource-Env11" API_KEY = "plIJ4ti4apyYLAPB" D_ENV = "AUTO" STATUS = "200" DESC = "" DNOTIF_TOPIC = "SDC-DISTR-NOTIF-TOPIC-AUTO" DSTATUS_TOPIC = "SDC-DISTR-STATUS-TOPIC-AUTO" ResponseCode=200 RequestID=null ElapsedTime=260 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.8) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:06:14.655Z InvocationID=99f7bc2c-012d-4c36-94ae-4bb853f5177a LogTimestamp=2021-08-11T16:06:14.915174Z ResponseDesc=OK Exiting. 2021-08-11T16:06:14.915Z [qtp2045766957-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc/v1/registerForDistribution HTTP/1.1 SC="200" 2021-08-11T16:06:16.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=988032ab-ffc8-456d-a064-07129d324953 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:06:16.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=988032ab-ffc8-456d-a064-07129d324953 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@54c4e333 2021-08-11T16:06:16.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=988032ab-ffc8-456d-a064-07129d324953 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:06:16.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=988032ab-ffc8-456d-a064-07129d324953 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:06:18.305Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:06:18.356Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:06:19.748Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12bc37d9-7426-4418-8f63-1370486da077 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:06:19.748Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12bc37d9-7426-4418-8f63-1370486da077 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6559ecff 2021-08-11T16:06:19.748Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12bc37d9-7426-4418-8f63-1370486da077 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:06:19.748Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12bc37d9-7426-4418-8f63-1370486da077 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:06:23.279Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aedcc97c-208d-4862-a10c-a8ac4514609a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:06:23.279Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aedcc97c-208d-4862-a10c-a8ac4514609a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@54216daa 2021-08-11T16:06:23.279Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aedcc97c-208d-4862-a10c-a8ac4514609a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:06:23.279Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aedcc97c-208d-4862-a10c-a8ac4514609a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:06:25.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0db1a2a8-3442-4157-b033-8f7767e4faee PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:06:25.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0db1a2a8-3442-4157-b033-8f7767e4faee PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@33f0a764 2021-08-11T16:06:25.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0db1a2a8-3442-4157-b033-8f7767e4faee PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:06:25.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0db1a2a8-3442-4157-b033-8f7767e4faee PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:06:28.962Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=30deeeb5-73c0-4f29-bc6a-c6db0be93fc7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:06:28.962Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=30deeeb5-73c0-4f29-bc6a-c6db0be93fc7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@405e0752 2021-08-11T16:06:28.962Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=30deeeb5-73c0-4f29-bc6a-c6db0be93fc7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:06:28.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=30deeeb5-73c0-4f29-bc6a-c6db0be93fc7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:06:30.378Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=31649 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/datatypes TargetServiceName=null EntryTimestamp=2021-08-11T16:05:58.728Z InvocationID=f0a94d88-637e-43e8-acaf-b704b65f8715 ErrorCode=500 LogTimestamp=2021-08-11T16:06:30.377572Z ResponseDesc=Created Exiting. 2021-08-11T16:06:30.378Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/datatypes HTTP/1.1 SC="201" 2021-08-11T16:06:30.450Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=3010d0d1-2679-4085-8570-b83ec990f5d0 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/uploadType/capability ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:06:30.449805Z InvocationID=fd1002c9-9713-4095-94f1-c63b4a09294d LogTimestamp=2021-08-11T16:06:30.449899Z Entering 2021-08-11T16:06:30.450Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=3010d0d1-2679-4085-8570-b83ec990f5d0 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/uploadType/capability ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:06:30.449805Z InvocationID=fd1002c9-9713-4095-94f1-c63b4a09294d LogTimestamp=2021-08-11T16:06:30.449899Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/uploadType/capability HTTP/1.1 2021-08-11T16:06:30.489Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : tosca.capabilities.Root to dao for create 2021-08-11T16:06:30.548Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : tosca.capabilities.Attachment to dao for create 2021-08-11T16:06:30.577Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : tosca.capabilities.Node to dao for create 2021-08-11T16:06:30.587Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : tosca.capabilities.Container to dao for create 2021-08-11T16:06:30.640Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : org.openecomp.capabilities.PortMirroring to dao for create 2021-08-11T16:06:30.653Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : tosca.capabilities.Endpoint to dao for create 2021-08-11T16:06:30.691Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : tosca.capabilities.DatabaseEndpoint to dao for create 2021-08-11T16:06:30.700Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : tosca.capabilities.Endpoint.Public to dao for create 2021-08-11T16:06:30.734Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : tosca.capabilities.Endpoint.Admin to dao for create 2021-08-11T16:06:30.756Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : tosca.capabilities.Endpoint.Database to dao for create 2021-08-11T16:06:30.776Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : tosca.capabilities.OperatingSystem to dao for create 2021-08-11T16:06:30.798Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : tosca.capabilities.Scalable to dao for create 2021-08-11T16:06:30.993Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : tosca.capabilities.network.Bindable to dao for create 2021-08-11T16:06:31.000Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : tosca.capabilities.Container.Docker to dao for create 2021-08-11T16:06:31.042Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : tosca.capabilities.network.Linkable to dao for create 2021-08-11T16:06:31.048Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : org.openecomp.capabilities.AllottedResource to dao for create 2021-08-11T16:06:31.060Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : tosca.capabilities.nfv.Metric to dao for create 2021-08-11T16:06:31.071Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : org.openecomp.capabilities.Metric to dao for create 2021-08-11T16:06:31.094Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : org.openecomp.capabilities.metric.Ceilometer to dao for create 2021-08-11T16:06:31.109Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : org.openecomp.capabilities.metric.SnmpPolling to dao for create 2021-08-11T16:06:31.121Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : org.openecomp.capabilities.metric.SnmpTrap to dao for create 2021-08-11T16:06:31.132Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : org.openecomp.capabilities.Forwarder to dao for create 2021-08-11T16:06:31.144Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : org.openecomp.capabilities.VLANAssignment to dao for create 2021-08-11T16:06:31.155Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : org.openecomp.capabilities.RoutingConfiguration to dao for create 2021-08-11T16:06:31.159Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : org.openecomp.capabilities.FabricConfiguration to dao for create 2021-08-11T16:06:31.167Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : tosca.capabilities.nfv.VirtualStorage to dao for create 2021-08-11T16:06:31.171Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : tosca.capabilities.nfv.VirtualCompute to dao for create 2021-08-11T16:06:31.203Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : tosca.capabilities.nfv.VirtualBindable to dao for create 2021-08-11T16:06:31.219Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : tosca.capabilities.nfv.VirtualLinkable to dao for create 2021-08-11T16:06:31.227Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : tosca.capabilities.nfv.ExtVirtualLinkable to dao for create 2021-08-11T16:06:31.242Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : tosca.capabilities.nfv.Forwarder to dao for create 2021-08-11T16:06:31.272Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : tosca.capabilities.nfv.ext.GuestOs to dao for create 2021-08-11T16:06:31.283Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : tosca.capabilities.nfv.ext.ImageFile to dao for create 2021-08-11T16:06:31.295Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : tosca.capabilities.nfv.ext.LocalAttachment to dao for create 2021-08-11T16:06:31.301Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : tosca.capabilities.nfv.ext.Compute.Container.Architecture to dao for create 2021-08-11T16:06:31.316Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send CAPABILITY_TYPE : tosca.capabilities.nfv.Forwarding to dao for create 2021-08-11T16:06:31.320Z [qtp2045766957-17] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p all CapabilityTypeDefinition(super=CapabilityTypeDataDefinition [uniqueId=tosca.capabilities.nfv.Forwarding, description=null, type=tosca.capabilities.nfv.Forwarding, validSourceTypes=null, version=null, creationTime=1628697991318, modificationTime=1628697991318], derivedFrom=tosca.capabilities.Root, properties=null) were created successfully!!! 2021-08-11T16:06:31.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b19ff39-65e9-49c2-b7fb-83be70b5d86e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:06:31.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b19ff39-65e9-49c2-b7fb-83be70b5d86e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6aca25b2 2021-08-11T16:06:31.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b19ff39-65e9-49c2-b7fb-83be70b5d86e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:06:31.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b19ff39-65e9-49c2-b7fb-83be70b5d86e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:06:33.309Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:06:33.346Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:06:33.371Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2919 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/capability EntryTimestamp=2021-08-11T16:06:30.451Z InvocationID=59096b79-7b1c-40d9-bdb2-a9870cd6b8f1 LogTimestamp=2021-08-11T16:06:33.370661Z ResponseDesc=Created Exiting. 2021-08-11T16:06:33.372Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/capability HTTP/1.1 SC="201" 2021-08-11T16:06:33.397Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=f91d7d8d-4007-4582-9389-e337b0e542c7 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/uploadType/relationship ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:06:33.397074Z InvocationID=3c1fa713-460a-4ad7-89dc-0d1ba52aa8d0 LogTimestamp=2021-08-11T16:06:33.397176Z Entering 2021-08-11T16:06:33.397Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=f91d7d8d-4007-4582-9389-e337b0e542c7 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/uploadType/relationship ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:06:33.397074Z InvocationID=3c1fa713-460a-4ad7-89dc-0d1ba52aa8d0 LogTimestamp=2021-08-11T16:06:33.397176Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/uploadType/relationship HTTP/1.1 2021-08-11T16:06:33.414Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2021-08-11T16:06:33.397Z InvocationID=48915e62-1955-42dd-a127-9c177006a360 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send RELATIONSHIP_TYPE : tosca.relationships.Root to dao for create 2021-08-11T16:06:34.055Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2021-08-11T16:06:33.397Z InvocationID=48915e62-1955-42dd-a127-9c177006a360 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send RELATIONSHIP_TYPE : tosca.relationships.ConnectsTo to dao for create 2021-08-11T16:06:34.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ea2c369-4474-492b-a9fe-225d47453f4a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:06:34.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ea2c369-4474-492b-a9fe-225d47453f4a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@35d9497c 2021-08-11T16:06:34.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ea2c369-4474-492b-a9fe-225d47453f4a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:06:34.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ea2c369-4474-492b-a9fe-225d47453f4a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:06:34.886Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2021-08-11T16:06:33.397Z InvocationID=48915e62-1955-42dd-a127-9c177006a360 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send RELATIONSHIP_TYPE : tosca.relationships.RoutesTo to dao for create 2021-08-11T16:06:35.543Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2021-08-11T16:06:33.397Z InvocationID=48915e62-1955-42dd-a127-9c177006a360 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send RELATIONSHIP_TYPE : tosca.relationships.AttachesTo to dao for create 2021-08-11T16:06:36.168Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2021-08-11T16:06:33.397Z InvocationID=48915e62-1955-42dd-a127-9c177006a360 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send RELATIONSHIP_TYPE : tosca.relationships.HostedOn to dao for create 2021-08-11T16:06:36.699Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2021-08-11T16:06:33.397Z InvocationID=48915e62-1955-42dd-a127-9c177006a360 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send RELATIONSHIP_TYPE : tosca.relationships.DependsOn to dao for create 2021-08-11T16:06:37.267Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2021-08-11T16:06:33.397Z InvocationID=48915e62-1955-42dd-a127-9c177006a360 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send RELATIONSHIP_TYPE : tosca.relationships.network.LinksTo to dao for create 2021-08-11T16:06:37.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=576f9a34-0b56-4019-a4ed-d6fb7e9e1a03 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:06:37.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=576f9a34-0b56-4019-a4ed-d6fb7e9e1a03 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3a41324 2021-08-11T16:06:37.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=576f9a34-0b56-4019-a4ed-d6fb7e9e1a03 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:06:37.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=576f9a34-0b56-4019-a4ed-d6fb7e9e1a03 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:06:37.843Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2021-08-11T16:06:33.397Z InvocationID=48915e62-1955-42dd-a127-9c177006a360 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send RELATIONSHIP_TYPE : tosca.relationships.network.BindsTo to dao for create 2021-08-11T16:06:38.386Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2021-08-11T16:06:33.397Z InvocationID=48915e62-1955-42dd-a127-9c177006a360 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send RELATIONSHIP_TYPE : org.openecomp.relationships.AttachesTo to dao for create 2021-08-11T16:06:38.979Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2021-08-11T16:06:33.397Z InvocationID=48915e62-1955-42dd-a127-9c177006a360 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send RELATIONSHIP_TYPE : org.openecomp.relationships.VolumeAttachesTo to dao for create 2021-08-11T16:06:39.666Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2021-08-11T16:06:33.397Z InvocationID=48915e62-1955-42dd-a127-9c177006a360 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send RELATIONSHIP_TYPE : org.openecomp.relationships.ForwardsTo to dao for create 2021-08-11T16:06:40.222Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2021-08-11T16:06:33.397Z InvocationID=48915e62-1955-42dd-a127-9c177006a360 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send RELATIONSHIP_TYPE : org.openecomp.relationships.AssignsTo to dao for create 2021-08-11T16:06:40.586Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c38258b1-0b02-43c7-b4d1-30ee238fb065 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:06:40.586Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c38258b1-0b02-43c7-b4d1-30ee238fb065 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3370e095 2021-08-11T16:06:40.586Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c38258b1-0b02-43c7-b4d1-30ee238fb065 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:06:40.586Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c38258b1-0b02-43c7-b4d1-30ee238fb065 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:06:40.755Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2021-08-11T16:06:33.397Z InvocationID=48915e62-1955-42dd-a127-9c177006a360 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send RELATIONSHIP_TYPE : org.openecomp.relationships.RoutesTo to dao for create 2021-08-11T16:06:41.308Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2021-08-11T16:06:33.397Z InvocationID=48915e62-1955-42dd-a127-9c177006a360 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send RELATIONSHIP_TYPE : tosca.relationships.nfv.VirtualBindsTo to dao for create 2021-08-11T16:06:41.865Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2021-08-11T16:06:33.397Z InvocationID=48915e62-1955-42dd-a127-9c177006a360 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send RELATIONSHIP_TYPE : tosca.relationships.nfv.VirtualLinksTo to dao for create 2021-08-11T16:06:42.437Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2021-08-11T16:06:33.397Z InvocationID=48915e62-1955-42dd-a127-9c177006a360 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send RELATIONSHIP_TYPE : tosca.relationships.nfv.AttachesTo to dao for create 2021-08-11T16:06:42.979Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2021-08-11T16:06:33.397Z InvocationID=48915e62-1955-42dd-a127-9c177006a360 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send RELATIONSHIP_TYPE : tosca.relationships.nfv.Vdu.AttachedTo to dao for create 2021-08-11T16:06:43.563Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2021-08-11T16:06:33.397Z InvocationID=48915e62-1955-42dd-a127-9c177006a360 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send RELATIONSHIP_TYPE : tosca.relationships.nfv.ForwardTo to dao for create 2021-08-11T16:06:43.694Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=710c3ecb-a4bb-46e9-ba62-8d2c9f43f867 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:06:43.694Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=710c3ecb-a4bb-46e9-ba62-8d2c9f43f867 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@82ca4fc 2021-08-11T16:06:43.694Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=710c3ecb-a4bb-46e9-ba62-8d2c9f43f867 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:06:43.695Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=710c3ecb-a4bb-46e9-ba62-8d2c9f43f867 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:06:44.190Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2021-08-11T16:06:33.397Z InvocationID=48915e62-1955-42dd-a127-9c177006a360 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send RELATIONSHIP_TYPE : tosca.relationships.nfv.ContainerBindsTo to dao for create 2021-08-11T16:06:44.735Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2021-08-11T16:06:33.397Z InvocationID=48915e62-1955-42dd-a127-9c177006a360 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p all RelationshipTypeDataDefinition [uniqueId=tosca.relationships.nfv.ContainerBindsTo, description=Represents an association relationship between Vdu.OsContainerGroup and VduCp node types, type=tosca.relationships.nfv.ContainerBindsTo, validSourceTypes=null, version=null, originUI=null, creationTime=1628698004191, modificationTime=1628698004191] [ derivedFrom=tosca.relationships.DependsOn, properties=null ] were created successfully!!! 2021-08-11T16:06:44.843Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=11444 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/relationship EntryTimestamp=2021-08-11T16:06:33.397Z InvocationID=48915e62-1955-42dd-a127-9c177006a360 LogTimestamp=2021-08-11T16:06:44.841650Z ResponseDesc=Created Exiting. 2021-08-11T16:06:44.843Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/relationship HTTP/1.1 SC="201" 2021-08-11T16:06:44.902Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=de66fa3b-7e51-420b-9214-83ab19d454e5 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/uploadType/interfaceLifecycle ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:06:44.895438Z InvocationID=984943f0-f2f0-495b-985a-9aef7a10a6c7 LogTimestamp=2021-08-11T16:06:44.895512Z Entering 2021-08-11T16:06:44.903Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=de66fa3b-7e51-420b-9214-83ab19d454e5 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/uploadType/interfaceLifecycle ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:06:44.895438Z InvocationID=984943f0-f2f0-495b-985a-9aef7a10a6c7 LogTimestamp=2021-08-11T16:06:44.895512Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/uploadType/interfaceLifecycle HTTP/1.1 2021-08-11T16:06:44.937Z [qtp2045766957-17] INFO o.o.s.b.c.i.InterfaceLifecycleTypeImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/interfaceLifecycle EntryTimestamp=2021-08-11T16:06:44.903Z InvocationID=2d78feda-f013-4c16-8d25-1bdc1fe146aa PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send interfaceDefinition tosca.interfaces.Root to dao for create 2021-08-11T16:06:45.464Z [qtp2045766957-17] INFO o.o.s.b.c.i.InterfaceLifecycleTypeImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/interfaceLifecycle EntryTimestamp=2021-08-11T16:06:44.903Z InvocationID=2d78feda-f013-4c16-8d25-1bdc1fe146aa PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send interfaceDefinition tosca.interfaces.nfv.vnf.lifecycle.Nfv to dao for create 2021-08-11T16:06:46.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bcf0397f-e482-48a2-8148-6c5b03401934 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:06:46.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bcf0397f-e482-48a2-8148-6c5b03401934 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@77c19e2c 2021-08-11T16:06:46.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bcf0397f-e482-48a2-8148-6c5b03401934 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:06:46.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bcf0397f-e482-48a2-8148-6c5b03401934 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:06:47.010Z [qtp2045766957-17] INFO o.o.s.b.c.i.InterfaceLifecycleTypeImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/interfaceLifecycle EntryTimestamp=2021-08-11T16:06:44.903Z InvocationID=2d78feda-f013-4c16-8d25-1bdc1fe146aa PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send interfaceDefinition tosca.interfaces.node.lifecycle.Standard to dao for create 2021-08-11T16:06:47.656Z [qtp2045766957-17] INFO o.o.s.b.c.i.InterfaceLifecycleTypeImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/interfaceLifecycle EntryTimestamp=2021-08-11T16:06:44.903Z InvocationID=2d78feda-f013-4c16-8d25-1bdc1fe146aa PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send interfaceDefinition tosca.interfaces.relationship.Configure to dao for create 2021-08-11T16:06:48.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:06:48.317Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:06:48.365Z [qtp2045766957-17] INFO o.o.s.b.c.i.InterfaceLifecycleTypeImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/interfaceLifecycle EntryTimestamp=2021-08-11T16:06:44.903Z InvocationID=2d78feda-f013-4c16-8d25-1bdc1fe146aa PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send interfaceDefinition tosca.interfaces.nfv.Vnflcm to dao for create 2021-08-11T16:06:49.290Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:06:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:06:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:06:50.150Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=394d746b-79ea-449e-b14a-b10e2d8fec01 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:06:50.150Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=394d746b-79ea-449e-b14a-b10e2d8fec01 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@54df9ec3 2021-08-11T16:06:50.150Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=394d746b-79ea-449e-b14a-b10e2d8fec01 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:06:50.150Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=394d746b-79ea-449e-b14a-b10e2d8fec01 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:06:50.650Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=1f78b199-9d74-4b95-9026-19da9257e1e2 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=sdnc ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.24.32 StatusCode=INPROGRESS RemoteHost=10.242.24.32 ServiceName=v1/artifactTypes ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:06:50.650262Z InvocationID=eeadab35-44e1-43b8-bd67-989cdfc818ec LogTimestamp=2021-08-11T16:06:50.650328Z Entering 2021-08-11T16:06:50.650Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=1f78b199-9d74-4b95-9026-19da9257e1e2 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=sdnc ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.24.32 StatusCode=INPROGRESS userId=null RemoteHost=10.242.24.32 ServiceName=v1/artifactTypes ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:06:50.650262Z InvocationID=eeadab35-44e1-43b8-bd67-989cdfc818ec LogTimestamp=2021-08-11T16:06:50.650328Z localAddr=10.242.24.223 remoteAddr=10.242.24.32 GET /sdc/v1/artifactTypes HTTP/1.1 2021-08-11T16:06:50.652Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=2 PartnerName=Apache-HttpClient/4.5.10 (Java/11.0.9.1) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/artifactTypes EntryTimestamp=2021-08-11T16:06:50.650Z InvocationID=5f0fbbfa-7ad9-4f14-8ad0-a8f9eed67f72 LogTimestamp=2021-08-11T16:06:50.652406Z ResponseDesc=OK Exiting. 2021-08-11T16:06:50.652Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc/v1/artifactTypes HTTP/1.1 SC="200" 2021-08-11T16:06:50.798Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=750cf95b-fbd3-4b16-a207-f4f9174dc71e InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=sdnc ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.24.32 StatusCode=INPROGRESS RemoteHost=10.242.24.32 ServiceName=v1/registerForDistribution ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:06:50.798481Z InvocationID=3e549e72-c8f5-47a6-8b6b-eabc1d7db528 LogTimestamp=2021-08-11T16:06:50.798545Z Entering 2021-08-11T16:06:50.798Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=750cf95b-fbd3-4b16-a207-f4f9174dc71e InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=sdnc ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.24.32 StatusCode=INPROGRESS userId=null RemoteHost=10.242.24.32 ServiceName=v1/registerForDistribution ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:06:50.798481Z InvocationID=3e549e72-c8f5-47a6-8b6b-eabc1d7db528 LogTimestamp=2021-08-11T16:06:50.798545Z localAddr=10.242.24.223 remoteAddr=10.242.24.32 POST /sdc/v1/registerForDistribution HTTP/1.1 2021-08-11T16:06:50.802Z [qtp2045766957-14] WARN c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:06:50.799Z InvocationID=d54ac399-6ff2-49fe-8626-213474ad7dbd PartnerName=Apache-HttpClient/4.5.10 (Java/11.0.9.1) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/producers/1ERWjNY915SsBhmA will send credentials over a clear channel. 2021-08-11T16:06:50.802Z [qtp2045766957-14] INFO c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:06:50.799Z InvocationID=d54ac399-6ff2-49fe-8626-213474ad7dbd PartnerName=Apache-HttpClient/4.5.10 (Java/11.0.9.1) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/producers/1ERWjNY915SsBhmA (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:06:50.820Z [qtp2045766957-14] INFO c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:06:50.799Z InvocationID=d54ac399-6ff2-49fe-8626-213474ad7dbd PartnerName=Apache-HttpClient/4.5.10 (Java/11.0.9.1) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p \t--> HTTP/1.1 200 OK 2021-08-11T16:06:50.825Z [qtp2045766957-14] WARN c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "1ERWjNY915SsBhmA" STATUS = "200" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:06:50.799Z InvocationID=d54ac399-6ff2-49fe-8626-213474ad7dbd PartnerName=Apache-HttpClient/4.5.10 (Java/11.0.9.1) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/consumers/1ERWjNY915SsBhmA will send credentials over a clear channel. 2021-08-11T16:06:50.825Z [qtp2045766957-14] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "1ERWjNY915SsBhmA" STATUS = "200" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:06:50.799Z InvocationID=d54ac399-6ff2-49fe-8626-213474ad7dbd PartnerName=Apache-HttpClient/4.5.10 (Java/11.0.9.1) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/consumers/1ERWjNY915SsBhmA (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:06:50.879Z [qtp2045766957-14] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "1ERWjNY915SsBhmA" STATUS = "200" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:06:50.799Z InvocationID=d54ac399-6ff2-49fe-8626-213474ad7dbd PartnerName=Apache-HttpClient/4.5.10 (Java/11.0.9.1) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p \t--> HTTP/1.1 200 OK 2021-08-11T16:06:50.963Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "DRegister" CONSUMER_ID = "sdc-COpenSource-Env11-sdnc-dockero" API_KEY = "1ERWjNY915SsBhmA" D_ENV = "AUTO" STATUS = "200" DESC = "" DNOTIF_TOPIC = "SDC-DISTR-NOTIF-TOPIC-AUTO" DSTATUS_TOPIC = "SDC-DISTR-STATUS-TOPIC-AUTO" ResponseCode=200 RequestID=null ElapsedTime=164 PartnerName=Apache-HttpClient/4.5.10 (Java/11.0.9.1) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:06:50.799Z InvocationID=d54ac399-6ff2-49fe-8626-213474ad7dbd LogTimestamp=2021-08-11T16:06:50.963582Z ResponseDesc=OK Exiting. 2021-08-11T16:06:50.964Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc/v1/registerForDistribution HTTP/1.1 SC="200" 2021-08-11T16:06:51.565Z [qtp2045766957-17] INFO o.o.s.b.c.i.InterfaceLifecycleTypeImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/interfaceLifecycle EntryTimestamp=2021-08-11T16:06:44.903Z InvocationID=2d78feda-f013-4c16-8d25-1bdc1fe146aa PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lifecycle types were created successfully!!! 2021-08-11T16:06:51.593Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=6689 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/interfaceLifecycle EntryTimestamp=2021-08-11T16:06:44.903Z InvocationID=2d78feda-f013-4c16-8d25-1bdc1fe146aa LogTimestamp=2021-08-11T16:06:51.592947Z ResponseDesc=Created Exiting. 2021-08-11T16:06:51.593Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/interfaceLifecycle HTTP/1.1 SC="201" 2021-08-11T16:06:51.666Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=4a864e63-3713-4d9d-bc6f-648fb0a47459 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/uploadType/categories ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:06:51.665896Z InvocationID=e8b97f8b-5dc7-44b6-b69f-7df5479ae5bd LogTimestamp=2021-08-11T16:06:51.665949Z Entering 2021-08-11T16:06:51.666Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=4a864e63-3713-4d9d-bc6f-648fb0a47459 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/uploadType/categories ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:06:51.665896Z InvocationID=e8b97f8b-5dc7-44b6-b69f-7df5479ae5bd LogTimestamp=2021-08-11T16:06:51.665949Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/uploadType/categories HTTP/1.1 2021-08-11T16:06:53.455Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d3c029b9-5d43-454d-8f65-a4e139ce8146 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:06:53.455Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d3c029b9-5d43-454d-8f65-a4e139ce8146 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@609fa885 2021-08-11T16:06:53.455Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d3c029b9-5d43-454d-8f65-a4e139ce8146 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:06:53.455Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d3c029b9-5d43-454d-8f65-a4e139ce8146 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:06:56.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c09235f-b211-4208-b0a6-1d30f811e807 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:06:56.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c09235f-b211-4208-b0a6-1d30f811e807 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@df9eb2e 2021-08-11T16:06:56.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c09235f-b211-4208-b0a6-1d30f811e807 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:06:56.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c09235f-b211-4208-b0a6-1d30f811e807 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:06:57.779Z [ApplicationDataTypeCacheThread-1] INFO o.o.s.b.m.c.ApplicationDataTypeCache RequestID=null Detected changes in the data types, updating the data type cache. 2021-08-11T16:06:58.967Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=baa0c338-e854-4957-90bc-220a4cfc1dcf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:06:58.967Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=baa0c338-e854-4957-90bc-220a4cfc1dcf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@76b22443 2021-08-11T16:06:58.967Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=baa0c338-e854-4957-90bc-220a4cfc1dcf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:06:58.967Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=baa0c338-e854-4957-90bc-220a4cfc1dcf PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:06:59.632Z [qtp2045766957-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=6f8c76a0-dd9d-49f2-b1a1-54a067929b90 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=clamp ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.24.220 StatusCode=INPROGRESS RemoteHost=10.242.24.220 ServiceName=v1/artifactTypes ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:06:59.632549Z InvocationID=2889f54c-c2a4-419b-ab56-a71ea5e2a130 LogTimestamp=2021-08-11T16:06:59.632622Z Entering 2021-08-11T16:06:59.632Z [qtp2045766957-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=6f8c76a0-dd9d-49f2-b1a1-54a067929b90 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=clamp ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.24.220 StatusCode=INPROGRESS userId=null RemoteHost=10.242.24.220 ServiceName=v1/artifactTypes ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:06:59.632549Z InvocationID=2889f54c-c2a4-419b-ab56-a71ea5e2a130 LogTimestamp=2021-08-11T16:06:59.632622Z localAddr=10.242.24.223 remoteAddr=10.242.24.220 GET /sdc/v1/artifactTypes HTTP/1.1 2021-08-11T16:06:59.634Z [qtp2045766957-11] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=200 RequestID=null ElapsedTime=1 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/artifactTypes EntryTimestamp=2021-08-11T16:06:59.633Z InvocationID=9674507a-ee03-4115-af93-0a1ed2c16171 LogTimestamp=2021-08-11T16:06:59.634470Z ResponseDesc=OK Exiting. 2021-08-11T16:06:59.636Z [qtp2045766957-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null GET /sdc/v1/artifactTypes HTTP/1.1 SC="200" 2021-08-11T16:06:59.985Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=45684946-8e09-4984-9f46-7e1a7754c03c InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=clamp ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.24.220 StatusCode=INPROGRESS RemoteHost=10.242.24.220 ServiceName=v1/registerForDistribution ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:06:59.984793Z InvocationID=14256746-9da8-4b85-a5d2-52f73a4b247c LogTimestamp=2021-08-11T16:06:59.984883Z Entering 2021-08-11T16:06:59.985Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=45684946-8e09-4984-9f46-7e1a7754c03c InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=clamp ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.24.220 StatusCode=INPROGRESS userId=null RemoteHost=10.242.24.220 ServiceName=v1/registerForDistribution ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:06:59.984793Z InvocationID=14256746-9da8-4b85-a5d2-52f73a4b247c LogTimestamp=2021-08-11T16:06:59.984883Z localAddr=10.242.24.223 remoteAddr=10.242.24.220 POST /sdc/v1/registerForDistribution HTTP/1.1 2021-08-11T16:06:59.989Z [qtp2045766957-16] WARN c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:06:59.985Z InvocationID=daaba851-a16f-45c0-9d7e-cb14f588bf0e PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/producers/bYIi0DneBH2QJCt1 will send credentials over a clear channel. 2021-08-11T16:06:59.989Z [qtp2045766957-16] INFO c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:06:59.985Z InvocationID=daaba851-a16f-45c0-9d7e-cb14f588bf0e PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-STATUS-TOPIC-AUTO/producers/bYIi0DneBH2QJCt1 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:07:00.009Z [qtp2045766957-16] INFO c.att.nsa.apiClient.http.HttpClient RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:06:59.985Z InvocationID=daaba851-a16f-45c0-9d7e-cb14f588bf0e PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p \t--> HTTP/1.1 200 OK 2021-08-11T16:07:00.013Z [qtp2045766957-16] WARN c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "bYIi0DneBH2QJCt1" STATUS = "200" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:06:59.985Z InvocationID=daaba851-a16f-45c0-9d7e-cb14f588bf0e PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/consumers/bYIi0DneBH2QJCt1 will send credentials over a clear channel. 2021-08-11T16:07:00.013Z [qtp2045766957-16] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "bYIi0DneBH2QJCt1" STATUS = "200" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:06:59.985Z InvocationID=daaba851-a16f-45c0-9d7e-cb14f588bf0e PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p PUT http://message-router.onap:3904/topics/SDC-DISTR-NOTIF-TOPIC-AUTO/consumers/bYIi0DneBH2QJCt1 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:07:00.033Z [qtp2045766957-16] INFO c.att.nsa.apiClient.http.HttpClient AuditMessage=ACTION = "AddKeyToTopicACL" D_ENV = "AUTO" TOPIC_NAME = "SDC-DISTR-STATUS-TOPIC-AUTO" ROLE = "PRODUCER" API_KEY = "bYIi0DneBH2QJCt1" STATUS = "200" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:06:59.985Z InvocationID=daaba851-a16f-45c0-9d7e-cb14f588bf0e PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p \t--> HTTP/1.1 200 OK 2021-08-11T16:07:00.037Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "DRegister" CONSUMER_ID = "clamp" API_KEY = "bYIi0DneBH2QJCt1" D_ENV = "AUTO" STATUS = "200" DESC = "" DNOTIF_TOPIC = "SDC-DISTR-NOTIF-TOPIC-AUTO" DSTATUS_TOPIC = "SDC-DISTR-STATUS-TOPIC-AUTO" ResponseCode=200 RequestID=null ElapsedTime=52 PartnerName=Apache-HttpClient/4.5.13 (Java/11.0.9.1) auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/registerForDistribution EntryTimestamp=2021-08-11T16:06:59.985Z InvocationID=daaba851-a16f-45c0-9d7e-cb14f588bf0e LogTimestamp=2021-08-11T16:07:00.037332Z ResponseDesc=OK Exiting. 2021-08-11T16:07:00.037Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc/v1/registerForDistribution HTTP/1.1 SC="200" 2021-08-11T16:07:01.359Z [ApplicationDataTypeCacheThread-1] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServiceName=ReplaceDataTypesCache TargetServiceName=null ErrorCode=500 TargetEntity=ReplaceDataTypesCache Internal flow error. Operation: Succeed to replace the data types cache 2021-08-11T16:07:01.693Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=002ac5dc-f804-4159-91e0-1e283b32588b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:07:01.693Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=002ac5dc-f804-4159-91e0-1e283b32588b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5e3385d0 2021-08-11T16:07:01.693Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=002ac5dc-f804-4159-91e0-1e283b32588b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:07:01.693Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=002ac5dc-f804-4159-91e0-1e283b32588b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:07:03.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:07:03.318Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:07:04.410Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:07:04.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86d8be52-103c-46ab-ab88-b1a7a959aab9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:07:04.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86d8be52-103c-46ab-ab88-b1a7a959aab9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4375c513 2021-08-11T16:07:04.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86d8be52-103c-46ab-ab88-b1a7a959aab9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:07:04.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86d8be52-103c-46ab-ab88-b1a7a959aab9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:07:07.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72190693-8158-43b1-89af-5bb7e3fffc69 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:07:07.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72190693-8158-43b1-89af-5bb7e3fffc69 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@117f6c8 2021-08-11T16:07:07.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72190693-8158-43b1-89af-5bb7e3fffc69 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:07:07.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72190693-8158-43b1-89af-5bb7e3fffc69 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:07:10.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e3297df2-1c96-4afd-8210-8b71572eb5af PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:07:10.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e3297df2-1c96-4afd-8210-8b71572eb5af PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@66604a95 2021-08-11T16:07:10.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e3297df2-1c96-4afd-8210-8b71572eb5af PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:07:10.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e3297df2-1c96-4afd-8210-8b71572eb5af PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:07:13.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41aaf18c-a077-4556-b592-70bd45440cfe PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:07:13.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41aaf18c-a077-4556-b592-70bd45440cfe PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@243f5e8c 2021-08-11T16:07:13.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41aaf18c-a077-4556-b592-70bd45440cfe PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:07:13.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41aaf18c-a077-4556-b592-70bd45440cfe PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:07:16.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=809bc863-dcf6-405b-9663-38db7b5e1c59 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:07:16.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=809bc863-dcf6-405b-9663-38db7b5e1c59 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@498f5a45 2021-08-11T16:07:16.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=809bc863-dcf6-405b-9663-38db7b5e1c59 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:07:16.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=809bc863-dcf6-405b-9663-38db7b5e1c59 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:07:18.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:07:18.318Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:07:19.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=af333d80-d92f-4711-bcc2-169df3a56a6b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:07:19.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=af333d80-d92f-4711-bcc2-169df3a56a6b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7710540a 2021-08-11T16:07:19.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=af333d80-d92f-4711-bcc2-169df3a56a6b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:07:19.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=af333d80-d92f-4711-bcc2-169df3a56a6b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:07:22.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e50ab87a-0d40-4229-83a3-39fdcaa2a339 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:07:22.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e50ab87a-0d40-4229-83a3-39fdcaa2a339 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@13409c9f 2021-08-11T16:07:22.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e50ab87a-0d40-4229-83a3-39fdcaa2a339 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:07:22.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e50ab87a-0d40-4229-83a3-39fdcaa2a339 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:07:25.078Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=33411 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/categories EntryTimestamp=2021-08-11T16:06:51.666Z InvocationID=625492d9-dce8-4204-b4a8-2b4143b1a1c4 LogTimestamp=2021-08-11T16:07:25.077432Z ResponseDesc=Created Exiting. 2021-08-11T16:07:25.079Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/categories HTTP/1.1 SC="201" 2021-08-11T16:07:25.118Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=f0e7f781-fadb-47f0-82bf-795b7fbf9dbd InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:07:25.118041Z InvocationID=407833d1-fe16-43d8-a65f-45c02dda5327 LogTimestamp=2021-08-11T16:07:25.118122Z Entering 2021-08-11T16:07:25.118Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=f0e7f781-fadb-47f0-82bf-795b7fbf9dbd InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:07:25.118041Z InvocationID=407833d1-fe16-43d8-a65f-45c02dda5327 LogTimestamp=2021-08-11T16:07:25.118122Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:07:25.948Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa61b4cd-11db-4f63-89fd-f089f89974d7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:07:25.948Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa61b4cd-11db-4f63-89fd-f089f89974d7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4d557b45 2021-08-11T16:07:25.948Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa61b4cd-11db-4f63-89fd-f089f89974d7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:07:25.948Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa61b4cd-11db-4f63-89fd-f089f89974d7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:07:26.186Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:07:25.119Z InvocationID=b6687441-6c4a-44d3-b8a4-4087ed9c3f76 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:07:26.186Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:07:25.119Z InvocationID=b6687441-6c4a-44d3-b8a4-4087ed9c3f76 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:07:26.186Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:07:25.119Z InvocationID=b6687441-6c4a-44d3-b8a4-4087ed9c3f76 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:07:26.186Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:07:25.119Z InvocationID=b6687441-6c4a-44d3-b8a4-4087ed9c3f76 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:07:26.276Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:07:25.119Z InvocationID=b6687441-6c4a-44d3-b8a4-4087ed9c3f76 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Root 2021-08-11T16:07:28.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db0b1bd7-5a39-4b30-af7e-ca72d8aab4bd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:07:28.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db0b1bd7-5a39-4b30-af7e-ca72d8aab4bd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7ebb5dd8 2021-08-11T16:07:28.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db0b1bd7-5a39-4b30-af7e-ca72d8aab4bd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:07:28.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db0b1bd7-5a39-4b30-af7e-ca72d8aab4bd PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:07:30.699Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Root" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "7229880c-c5c0-43fa-862e-3660a1ae1083" INVARIANT_UUID = "03fe0591-3ac6-4ad8-b97d-4f08f71c5111" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.Root" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7229880c-c5c0-43fa-862e-3660a1ae1083 EntryTimestamp=2021-08-11T16:07:25.119Z InvocationID=b6687441-6c4a-44d3-b8a4-4087ed9c3f76 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id bf1155d2-8819-44aa-9047-1f1f5b16c85e 2021-08-11T16:07:31.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e7a5f983-73f7-4edb-a9be-6d41a9d0b101 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:07:31.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e7a5f983-73f7-4edb-a9be-6d41a9d0b101 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@b7ef601 2021-08-11T16:07:31.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e7a5f983-73f7-4edb-a9be-6d41a9d0b101 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:07:31.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e7a5f983-73f7-4edb-a9be-6d41a9d0b101 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:07:32.970Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Root" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "7229880c-c5c0-43fa-862e-3660a1ae1083" INVARIANT_UUID = "03fe0591-3ac6-4ad8-b97d-4f08f71c5111" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "bf1155d2-8819-44aa-9047-1f1f5b16c85e.assettoscatemplate" CURR_ARTIFACT_UUID = "bf1155d2-8819-44aa-9047-1f1f5b16c85e.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Root-template.yml,0,bf1155d2-8819-44aa-9047-1f1f5b16c85e.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=7851 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bbb581b7-b4f5-4ed2-b1cd-512655be8314 EntryTimestamp=2021-08-11T16:07:25.119Z InvocationID=b6687441-6c4a-44d3-b8a4-4087ed9c3f76 LogTimestamp=2021-08-11T16:07:32.970238Z ResponseDesc=Created Exiting. 2021-08-11T16:07:32.970Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:07:32.983Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=886eaac4-9074-4dcd-9639-f72d299b8b7e InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:07:32.983685Z InvocationID=a4f47369-bb97-4d88-9a37-f97a77cf81ae LogTimestamp=2021-08-11T16:07:32.983733Z Entering 2021-08-11T16:07:32.984Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=886eaac4-9074-4dcd-9639-f72d299b8b7e InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:07:32.983685Z InvocationID=a4f47369-bb97-4d88-9a37-f97a77cf81ae LogTimestamp=2021-08-11T16:07:32.983733Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:07:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:07:33.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:07:33.439Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:07:32.984Z InvocationID=75462429-cd3c-46a4-9dd5-cba6c943cb58 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:07:33.440Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:07:32.984Z InvocationID=75462429-cd3c-46a4-9dd5-cba6c943cb58 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:07:33.440Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:07:32.984Z InvocationID=75462429-cd3c-46a4-9dd5-cba6c943cb58 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:07:33.440Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:07:32.984Z InvocationID=75462429-cd3c-46a4-9dd5-cba6c943cb58 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:07:33.579Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:07:32.984Z InvocationID=75462429-cd3c-46a4-9dd5-cba6c943cb58 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Compute 2021-08-11T16:07:34.752Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=51397f60-c737-4db4-a39d-3f58f094f7a8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:07:34.752Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=51397f60-c737-4db4-a39d-3f58f094f7a8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1c4d2795 2021-08-11T16:07:34.752Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=51397f60-c737-4db4-a39d-3f58f094f7a8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:07:34.752Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=51397f60-c737-4db4-a39d-3f58f094f7a8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:07:35.991Z [qtp2045766957-14] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "38a46f9a-96d4-425a-807e-22118a23730a" INVARIANT_UUID = "49cf4eb7-b4e3-428d-bd25-6deb05237b5a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.Compute" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=38a46f9a-96d4-425a-807e-22118a23730a EntryTimestamp=2021-08-11T16:07:32.984Z InvocationID=75462429-cd3c-46a4-9dd5-cba6c943cb58 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:07:35.993Z [qtp2045766957-14] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "38a46f9a-96d4-425a-807e-22118a23730a" INVARIANT_UUID = "49cf4eb7-b4e3-428d-bd25-6deb05237b5a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.Compute" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=38a46f9a-96d4-425a-807e-22118a23730a TargetServiceName=null EntryTimestamp=2021-08-11T16:07:32.984Z InvocationID=75462429-cd3c-46a4-9dd5-cba6c943cb58 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:07:36.243Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "38a46f9a-96d4-425a-807e-22118a23730a" INVARIANT_UUID = "49cf4eb7-b4e3-428d-bd25-6deb05237b5a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.Compute" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=38a46f9a-96d4-425a-807e-22118a23730a TargetServiceName=null EntryTimestamp=2021-08-11T16:07:32.984Z InvocationID=75462429-cd3c-46a4-9dd5-cba6c943cb58 ErrorCode=500 lock resource with id 084c40db-cf25-4d7e-b0aa-33eea67875f8 2021-08-11T16:07:37.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=05d2aee9-016d-4fdf-8629-96d4aa295784 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:07:37.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=05d2aee9-016d-4fdf-8629-96d4aa295784 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7a24f80a 2021-08-11T16:07:37.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=05d2aee9-016d-4fdf-8629-96d4aa295784 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:07:37.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=05d2aee9-016d-4fdf-8629-96d4aa295784 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:07:37.965Z [qtp2045766957-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "38a46f9a-96d4-425a-807e-22118a23730a" INVARIANT_UUID = "49cf4eb7-b4e3-428d-bd25-6deb05237b5a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "084c40db-cf25-4d7e-b0aa-33eea67875f8.assettoscatemplate" CURR_ARTIFACT_UUID = "084c40db-cf25-4d7e-b0aa-33eea67875f8.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Compute-template.yml,0,084c40db-cf25-4d7e-b0aa-33eea67875f8.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=161ee141-65fe-41a2-9c5b-c9dd6bca0dcc TargetServiceName=null EntryTimestamp=2021-08-11T16:07:32.984Z InvocationID=75462429-cd3c-46a4-9dd5-cba6c943cb58 ErrorCode=500 Starting to create Csar for component Compute 2021-08-11T16:07:38.374Z [qtp2045766957-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "38a46f9a-96d4-425a-807e-22118a23730a" INVARIANT_UUID = "49cf4eb7-b4e3-428d-bd25-6deb05237b5a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "084c40db-cf25-4d7e-b0aa-33eea67875f8.assettoscatemplate" CURR_ARTIFACT_UUID = "084c40db-cf25-4d7e-b0aa-33eea67875f8.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Compute-template.yml,0,084c40db-cf25-4d7e-b0aa-33eea67875f8.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=161ee141-65fe-41a2-9c5b-c9dd6bca0dcc TargetServiceName=null EntryTimestamp=2021-08-11T16:07:32.984Z InvocationID=75462429-cd3c-46a4-9dd5-cba6c943cb58 ErrorCode=500 Ended create Csar for component Compute 2021-08-11T16:07:38.378Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "38a46f9a-96d4-425a-807e-22118a23730a" INVARIANT_UUID = "49cf4eb7-b4e3-428d-bd25-6deb05237b5a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "084c40db-cf25-4d7e-b0aa-33eea67875f8.assettoscatemplate" CURR_ARTIFACT_UUID = "084c40db-cf25-4d7e-b0aa-33eea67875f8.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Compute-template.yml,0,084c40db-cf25-4d7e-b0aa-33eea67875f8.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=161ee141-65fe-41a2-9c5b-c9dd6bca0dcc TargetServiceName=null EntryTimestamp=2021-08-11T16:07:32.984Z InvocationID=75462429-cd3c-46a4-9dd5-cba6c943cb58 ErrorCode=500 lock resource with id 084c40db-cf25-4d7e-b0aa-33eea67875f8 2021-08-11T16:07:40.206Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "38a46f9a-96d4-425a-807e-22118a23730a" INVARIANT_UUID = "49cf4eb7-b4e3-428d-bd25-6deb05237b5a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "084c40db-cf25-4d7e-b0aa-33eea67875f8.assettoscacsar" CURR_ARTIFACT_UUID = "084c40db-cf25-4d7e-b0aa-33eea67875f8.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Compute-csar.csar,0,084c40db-cf25-4d7e-b0aa-33eea67875f8.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=7222 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=6d8808f8-418c-48ee-b968-9dd6e8eac0e4 TargetServiceName=null EntryTimestamp=2021-08-11T16:07:32.984Z InvocationID=75462429-cd3c-46a4-9dd5-cba6c943cb58 ErrorCode=500 LogTimestamp=2021-08-11T16:07:40.206378Z ResponseDesc=Created Exiting. 2021-08-11T16:07:40.207Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:07:40.267Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=ff2be6ab-471e-4fc5-b645-f26c34af4a63 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:07:40.267515Z InvocationID=05119714-b882-4399-9df1-582f19ed12c0 LogTimestamp=2021-08-11T16:07:40.267608Z Entering 2021-08-11T16:07:40.268Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=ff2be6ab-471e-4fc5-b645-f26c34af4a63 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:07:40.267515Z InvocationID=05119714-b882-4399-9df1-582f19ed12c0 LogTimestamp=2021-08-11T16:07:40.267608Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:07:41.066Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd79191f-2750-4141-9bf1-a05bc648ff28 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:07:41.066Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd79191f-2750-4141-9bf1-a05bc648ff28 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5e552f0b 2021-08-11T16:07:41.066Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd79191f-2750-4141-9bf1-a05bc648ff28 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:07:41.066Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd79191f-2750-4141-9bf1-a05bc648ff28 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:07:41.879Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:07:40.268Z InvocationID=5c56f3f4-e2c9-4b5d-bd4d-ee4c28812231 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:07:41.879Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:07:40.268Z InvocationID=5c56f3f4-e2c9-4b5d-bd4d-ee4c28812231 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:07:41.879Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:07:40.268Z InvocationID=5c56f3f4-e2c9-4b5d-bd4d-ee4c28812231 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:07:41.879Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:07:40.268Z InvocationID=5c56f3f4-e2c9-4b5d-bd4d-ee4c28812231 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:07:41.973Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:07:40.268Z InvocationID=5c56f3f4-e2c9-4b5d-bd4d-ee4c28812231 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Softwarecomponent 2021-08-11T16:07:44.148Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52cf1f04-ea39-4cf3-84c8-1d2b17792f43 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:07:44.148Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52cf1f04-ea39-4cf3-84c8-1d2b17792f43 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@22e7043e 2021-08-11T16:07:44.148Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52cf1f04-ea39-4cf3-84c8-1d2b17792f43 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:07:44.148Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52cf1f04-ea39-4cf3-84c8-1d2b17792f43 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:07:44.702Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "SoftwareComponent" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "79e3562d-4e61-4612-8c97-cdd7ec477138" INVARIANT_UUID = "87f6a6b7-68b2-41d6-b1a5-18a3c6350fb8" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.SoftwareComponent" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=79e3562d-4e61-4612-8c97-cdd7ec477138 EntryTimestamp=2021-08-11T16:07:40.268Z InvocationID=5c56f3f4-e2c9-4b5d-bd4d-ee4c28812231 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id 297cc5ca-883b-4002-a966-431aa8984dd1 2021-08-11T16:07:46.435Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "SoftwareComponent" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "79e3562d-4e61-4612-8c97-cdd7ec477138" INVARIANT_UUID = "87f6a6b7-68b2-41d6-b1a5-18a3c6350fb8" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "297cc5ca-883b-4002-a966-431aa8984dd1.assettoscatemplate" CURR_ARTIFACT_UUID = "297cc5ca-883b-4002-a966-431aa8984dd1.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Softwarecomponent-template.yml,0,297cc5ca-883b-4002-a966-431aa8984dd1.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=6167 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=fff7c927-e717-4a26-83c5-ea8799fad5d9 EntryTimestamp=2021-08-11T16:07:40.268Z InvocationID=5c56f3f4-e2c9-4b5d-bd4d-ee4c28812231 LogTimestamp=2021-08-11T16:07:46.435350Z ResponseDesc=Created Exiting. 2021-08-11T16:07:46.436Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:07:46.455Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=0df01649-6280-463f-9018-e87e6e1928e8 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:07:46.454713Z InvocationID=7cf99758-a214-47ee-87e6-42266587f62f LogTimestamp=2021-08-11T16:07:46.454862Z Entering 2021-08-11T16:07:46.455Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=0df01649-6280-463f-9018-e87e6e1928e8 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:07:46.454713Z InvocationID=7cf99758-a214-47ee-87e6-42266587f62f LogTimestamp=2021-08-11T16:07:46.454862Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:07:46.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3f24a2a-7014-4d58-a140-d6a51975fa18 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:07:46.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3f24a2a-7014-4d58-a140-d6a51975fa18 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@57e3ee78 2021-08-11T16:07:46.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3f24a2a-7014-4d58-a140-d6a51975fa18 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:07:46.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3f24a2a-7014-4d58-a140-d6a51975fa18 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:07:47.062Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:07:46.455Z InvocationID=8715a426-ab20-42f0-8676-a109cdd594a6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:07:47.062Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:07:46.455Z InvocationID=8715a426-ab20-42f0-8676-a109cdd594a6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:07:47.062Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:07:46.455Z InvocationID=8715a426-ab20-42f0-8676-a109cdd594a6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:07:47.062Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:07:46.455Z InvocationID=8715a426-ab20-42f0-8676-a109cdd594a6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:07:47.143Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:07:46.455Z InvocationID=8715a426-ab20-42f0-8676-a109cdd594a6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Webserver 2021-08-11T16:07:48.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:07:48.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:07:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:07:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:07:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:07:50.254Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "WebServer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "9f18c962-d87a-4378-a23f-38bf5bf8492a" INVARIANT_UUID = "4a946c63-ad6d-4fbc-93c1-8b53bafd5955" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.WebServer" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9f18c962-d87a-4378-a23f-38bf5bf8492a EntryTimestamp=2021-08-11T16:07:46.455Z InvocationID=8715a426-ab20-42f0-8676-a109cdd594a6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id 4b3b970f-5ace-4d4e-969a-b12fd150a000 2021-08-11T16:07:50.451Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b311ccae-7ddd-4cbd-bd71-d94a781009ed PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:07:50.452Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b311ccae-7ddd-4cbd-bd71-d94a781009ed PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@adab826 2021-08-11T16:07:50.452Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b311ccae-7ddd-4cbd-bd71-d94a781009ed PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:07:50.452Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b311ccae-7ddd-4cbd-bd71-d94a781009ed PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:07:52.185Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "WebServer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "9f18c962-d87a-4378-a23f-38bf5bf8492a" INVARIANT_UUID = "4a946c63-ad6d-4fbc-93c1-8b53bafd5955" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "4b3b970f-5ace-4d4e-969a-b12fd150a000.assettoscatemplate" CURR_ARTIFACT_UUID = "4b3b970f-5ace-4d4e-969a-b12fd150a000.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Webserver-template.yml,0,4b3b970f-5ace-4d4e-969a-b12fd150a000.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=5729 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=38815038-e178-4eac-bc0b-a34c30168047 EntryTimestamp=2021-08-11T16:07:46.455Z InvocationID=8715a426-ab20-42f0-8676-a109cdd594a6 LogTimestamp=2021-08-11T16:07:52.184892Z ResponseDesc=Created Exiting. 2021-08-11T16:07:52.185Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:07:52.203Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=97848b7a-daaa-42e0-8f27-9119a174a473 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:07:52.203530Z InvocationID=bf385f9f-8261-4ace-8c8a-e20df5771c69 LogTimestamp=2021-08-11T16:07:52.203605Z Entering 2021-08-11T16:07:52.204Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=97848b7a-daaa-42e0-8f27-9119a174a473 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:07:52.203530Z InvocationID=bf385f9f-8261-4ace-8c8a-e20df5771c69 LogTimestamp=2021-08-11T16:07:52.203605Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:07:52.370Z [ComponentsCleanThread-1] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null lock resource with name DELETE_LOCKER 2021-08-11T16:07:53.048Z [ComponentsCleanThread-1] INFO o.o.s.b.c.s.ComponentsCleanBusinessLogic RequestID=null Lock cleanup operation is done with result = OK 2021-08-11T16:07:53.460Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dbad2d89-720f-411d-9a9d-0b2275203d13 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:07:53.460Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dbad2d89-720f-411d-9a9d-0b2275203d13 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7d80e301 2021-08-11T16:07:53.460Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dbad2d89-720f-411d-9a9d-0b2275203d13 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:07:53.460Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dbad2d89-720f-411d-9a9d-0b2275203d13 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:07:53.985Z [ComponentsCleanThread-1] INFO o.o.s.b.c.s.ComponentsCleanBusinessLogic RequestID=null Unlock cleanup operation is done with result = OK 2021-08-11T16:07:54.166Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:07:52.204Z InvocationID=5f1cceb8-f6dd-4ef1-8fe3-de13faf50157 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:07:54.166Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:07:52.204Z InvocationID=5f1cceb8-f6dd-4ef1-8fe3-de13faf50157 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:07:54.166Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:07:52.204Z InvocationID=5f1cceb8-f6dd-4ef1-8fe3-de13faf50157 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:07:54.166Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:07:52.204Z InvocationID=5f1cceb8-f6dd-4ef1-8fe3-de13faf50157 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:07:54.292Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:07:52.204Z InvocationID=5f1cceb8-f6dd-4ef1-8fe3-de13faf50157 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Webapplication 2021-08-11T16:07:56.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75cdb765-581e-41f6-b63e-a6ec6653d3f6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:07:56.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75cdb765-581e-41f6-b63e-a6ec6653d3f6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@67dcde7b 2021-08-11T16:07:56.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75cdb765-581e-41f6-b63e-a6ec6653d3f6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:07:56.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75cdb765-581e-41f6-b63e-a6ec6653d3f6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:07:56.911Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "WebApplication" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "9ec33768-76c6-4ef5-b664-993300343deb" INVARIANT_UUID = "a91a325e-d243-48b7-8ec9-b83a15a274b3" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.WebApplication" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9ec33768-76c6-4ef5-b664-993300343deb EntryTimestamp=2021-08-11T16:07:52.204Z InvocationID=5f1cceb8-f6dd-4ef1-8fe3-de13faf50157 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id 0ad09ca4-b54a-4c4f-afc5-704432a6690f 2021-08-11T16:07:58.811Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "WebApplication" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "9ec33768-76c6-4ef5-b664-993300343deb" INVARIANT_UUID = "a91a325e-d243-48b7-8ec9-b83a15a274b3" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "0ad09ca4-b54a-4c4f-afc5-704432a6690f.assettoscatemplate" CURR_ARTIFACT_UUID = "0ad09ca4-b54a-4c4f-afc5-704432a6690f.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Webapplication-template.yml,0,0ad09ca4-b54a-4c4f-afc5-704432a6690f.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=6607 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f61747be-cc10-4d1e-95a3-a59d8e8adb6f EntryTimestamp=2021-08-11T16:07:52.204Z InvocationID=5f1cceb8-f6dd-4ef1-8fe3-de13faf50157 LogTimestamp=2021-08-11T16:07:58.811122Z ResponseDesc=Created Exiting. 2021-08-11T16:07:58.811Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:07:58.872Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=5ae0e426-4fca-4d00-8009-2b5dec4c79fe InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:07:58.870461Z InvocationID=4f2405f3-9161-487d-9cbc-675e33a6430a LogTimestamp=2021-08-11T16:07:58.870588Z Entering 2021-08-11T16:07:58.872Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=5ae0e426-4fca-4d00-8009-2b5dec4c79fe InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:07:58.870461Z InvocationID=4f2405f3-9161-487d-9cbc-675e33a6430a LogTimestamp=2021-08-11T16:07:58.870588Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:07:58.988Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=016782e1-fe37-4f79-8fc1-be1f8e622872 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:07:58.988Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=016782e1-fe37-4f79-8fc1-be1f8e622872 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2f847cd6 2021-08-11T16:07:58.989Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=016782e1-fe37-4f79-8fc1-be1f8e622872 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:07:58.989Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=016782e1-fe37-4f79-8fc1-be1f8e622872 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:07:59.262Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:07:58.872Z InvocationID=0c1bf008-c1b5-41bd-9c76-c72f6f31fc65 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:07:59.262Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:07:58.872Z InvocationID=0c1bf008-c1b5-41bd-9c76-c72f6f31fc65 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:07:59.262Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:07:58.872Z InvocationID=0c1bf008-c1b5-41bd-9c76-c72f6f31fc65 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:07:59.262Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:07:58.872Z InvocationID=0c1bf008-c1b5-41bd-9c76-c72f6f31fc65 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:07:59.289Z [qtp2045766957-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:07:58.872Z InvocationID=0c1bf008-c1b5-41bd-9c76-c72f6f31fc65 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Dbms 2021-08-11T16:08:00.847Z [qtp2045766957-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "DBMS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "a47a5acc-67ea-4cce-844e-98b2ebac58da" INVARIANT_UUID = "49c4323c-3421-4b2e-a85b-c47bf896de01" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.DBMS" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a47a5acc-67ea-4cce-844e-98b2ebac58da EntryTimestamp=2021-08-11T16:07:58.872Z InvocationID=0c1bf008-c1b5-41bd-9c76-c72f6f31fc65 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id 337df1c6-ffd3-44f8-80df-0b8155d37054 2021-08-11T16:08:01.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=53566737-4f2e-42da-982a-e07644a69cd5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:08:01.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=53566737-4f2e-42da-982a-e07644a69cd5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7040ea88 2021-08-11T16:08:01.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=53566737-4f2e-42da-982a-e07644a69cd5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:08:01.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=53566737-4f2e-42da-982a-e07644a69cd5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:08:02.471Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "DBMS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "a47a5acc-67ea-4cce-844e-98b2ebac58da" INVARIANT_UUID = "49c4323c-3421-4b2e-a85b-c47bf896de01" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "337df1c6-ffd3-44f8-80df-0b8155d37054.assettoscatemplate" CURR_ARTIFACT_UUID = "337df1c6-ffd3-44f8-80df-0b8155d37054.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Dbms-template.yml,0,337df1c6-ffd3-44f8-80df-0b8155d37054.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=3599 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a50f4295-1d9e-4b62-831a-5f2ccc7f80b1 EntryTimestamp=2021-08-11T16:07:58.872Z InvocationID=0c1bf008-c1b5-41bd-9c76-c72f6f31fc65 LogTimestamp=2021-08-11T16:08:02.471398Z ResponseDesc=Created Exiting. 2021-08-11T16:08:02.472Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:08:02.488Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=ab8473e8-9d38-4620-b2a9-e351c8487f7c InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:08:02.487958Z InvocationID=4c6b9468-de7d-4b47-9db7-7b316f3ce4a4 LogTimestamp=2021-08-11T16:08:02.488029Z Entering 2021-08-11T16:08:02.488Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=ab8473e8-9d38-4620-b2a9-e351c8487f7c InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:08:02.487958Z InvocationID=4c6b9468-de7d-4b47-9db7-7b316f3ce4a4 LogTimestamp=2021-08-11T16:08:02.488029Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:08:02.757Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:02.488Z InvocationID=67773fe0-a5f8-43a5-a636-6a0073edd17d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:02.757Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:02.488Z InvocationID=67773fe0-a5f8-43a5-a636-6a0073edd17d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:02.757Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:02.488Z InvocationID=67773fe0-a5f8-43a5-a636-6a0073edd17d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:02.757Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:02.488Z InvocationID=67773fe0-a5f8-43a5-a636-6a0073edd17d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:02.797Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:02.488Z InvocationID=67773fe0-a5f8-43a5-a636-6a0073edd17d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Database 2021-08-11T16:08:03.309Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:08:03.317Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:08:04.184Z [qtp2045766957-18] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Database" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "1330992f-b275-4f37-b60f-14d79c453952" INVARIANT_UUID = "74cc6d76-9fd1-4661-a9cf-f15a95b2c4ea" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.Database" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1330992f-b275-4f37-b60f-14d79c453952 EntryTimestamp=2021-08-11T16:08:02.488Z InvocationID=67773fe0-a5f8-43a5-a636-6a0073edd17d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:08:04.185Z [qtp2045766957-18] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Database" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "1330992f-b275-4f37-b60f-14d79c453952" INVARIANT_UUID = "74cc6d76-9fd1-4661-a9cf-f15a95b2c4ea" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.Database" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1330992f-b275-4f37-b60f-14d79c453952 TargetServiceName=null EntryTimestamp=2021-08-11T16:08:02.488Z InvocationID=67773fe0-a5f8-43a5-a636-6a0073edd17d ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:08:04.274Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:08:04.687Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Database" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "1330992f-b275-4f37-b60f-14d79c453952" INVARIANT_UUID = "74cc6d76-9fd1-4661-a9cf-f15a95b2c4ea" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.Database" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1330992f-b275-4f37-b60f-14d79c453952 TargetServiceName=null EntryTimestamp=2021-08-11T16:08:02.488Z InvocationID=67773fe0-a5f8-43a5-a636-6a0073edd17d ErrorCode=500 lock resource with id da1fb56a-deb7-4695-a616-4ca2e75e9db0 2021-08-11T16:08:05.002Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3d9af5d9-2a6b-4a10-9f72-7cbcbfc1b536 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:08:05.002Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3d9af5d9-2a6b-4a10-9f72-7cbcbfc1b536 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7e20028 2021-08-11T16:08:05.002Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3d9af5d9-2a6b-4a10-9f72-7cbcbfc1b536 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:08:05.002Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3d9af5d9-2a6b-4a10-9f72-7cbcbfc1b536 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:08:06.301Z [qtp2045766957-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Database" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "1330992f-b275-4f37-b60f-14d79c453952" INVARIANT_UUID = "74cc6d76-9fd1-4661-a9cf-f15a95b2c4ea" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "da1fb56a-deb7-4695-a616-4ca2e75e9db0.assettoscatemplate" CURR_ARTIFACT_UUID = "da1fb56a-deb7-4695-a616-4ca2e75e9db0.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Database-template.yml,0,da1fb56a-deb7-4695-a616-4ca2e75e9db0.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=4970b76b-7dd6-4ff6-8e71-a17625637f49 TargetServiceName=null EntryTimestamp=2021-08-11T16:08:02.488Z InvocationID=67773fe0-a5f8-43a5-a636-6a0073edd17d ErrorCode=500 Starting to create Csar for component Database 2021-08-11T16:08:06.532Z [qtp2045766957-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Database" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "1330992f-b275-4f37-b60f-14d79c453952" INVARIANT_UUID = "74cc6d76-9fd1-4661-a9cf-f15a95b2c4ea" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "da1fb56a-deb7-4695-a616-4ca2e75e9db0.assettoscatemplate" CURR_ARTIFACT_UUID = "da1fb56a-deb7-4695-a616-4ca2e75e9db0.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Database-template.yml,0,da1fb56a-deb7-4695-a616-4ca2e75e9db0.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=4970b76b-7dd6-4ff6-8e71-a17625637f49 TargetServiceName=null EntryTimestamp=2021-08-11T16:08:02.488Z InvocationID=67773fe0-a5f8-43a5-a636-6a0073edd17d ErrorCode=500 Ended create Csar for component Database 2021-08-11T16:08:06.533Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Database" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "1330992f-b275-4f37-b60f-14d79c453952" INVARIANT_UUID = "74cc6d76-9fd1-4661-a9cf-f15a95b2c4ea" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "da1fb56a-deb7-4695-a616-4ca2e75e9db0.assettoscatemplate" CURR_ARTIFACT_UUID = "da1fb56a-deb7-4695-a616-4ca2e75e9db0.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Database-template.yml,0,da1fb56a-deb7-4695-a616-4ca2e75e9db0.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=4970b76b-7dd6-4ff6-8e71-a17625637f49 TargetServiceName=null EntryTimestamp=2021-08-11T16:08:02.488Z InvocationID=67773fe0-a5f8-43a5-a636-6a0073edd17d ErrorCode=500 lock resource with id da1fb56a-deb7-4695-a616-4ca2e75e9db0 2021-08-11T16:08:07.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=26a94188-0816-48ac-905c-b1449850499d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:08:07.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=26a94188-0816-48ac-905c-b1449850499d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7001aef6 2021-08-11T16:08:07.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=26a94188-0816-48ac-905c-b1449850499d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:08:07.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=26a94188-0816-48ac-905c-b1449850499d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:08:08.222Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Database" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "1330992f-b275-4f37-b60f-14d79c453952" INVARIANT_UUID = "74cc6d76-9fd1-4661-a9cf-f15a95b2c4ea" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "da1fb56a-deb7-4695-a616-4ca2e75e9db0.assettoscacsar" CURR_ARTIFACT_UUID = "da1fb56a-deb7-4695-a616-4ca2e75e9db0.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Database-csar.csar,0,da1fb56a-deb7-4695-a616-4ca2e75e9db0.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5734 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=08954da8-783d-4ea9-bce9-e2c40ac84220 TargetServiceName=null EntryTimestamp=2021-08-11T16:08:02.488Z InvocationID=67773fe0-a5f8-43a5-a636-6a0073edd17d ErrorCode=500 LogTimestamp=2021-08-11T16:08:08.222499Z ResponseDesc=Created Exiting. 2021-08-11T16:08:08.223Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:08:08.256Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=766d4096-ca9c-4fc1-a8ef-6f8df980eda4 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:08:08.255630Z InvocationID=6cc97218-397f-42a6-913b-c864bbae3616 LogTimestamp=2021-08-11T16:08:08.255731Z Entering 2021-08-11T16:08:08.256Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=766d4096-ca9c-4fc1-a8ef-6f8df980eda4 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:08:08.255630Z InvocationID=6cc97218-397f-42a6-913b-c864bbae3616 LogTimestamp=2021-08-11T16:08:08.255731Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:08:08.501Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:08.256Z InvocationID=543a2e2c-8de5-412a-b846-909bc5f1eac8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:08.501Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:08.256Z InvocationID=543a2e2c-8de5-412a-b846-909bc5f1eac8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:08.501Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:08.256Z InvocationID=543a2e2c-8de5-412a-b846-909bc5f1eac8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:08.501Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:08.256Z InvocationID=543a2e2c-8de5-412a-b846-909bc5f1eac8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:08.550Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:08.256Z InvocationID=543a2e2c-8de5-412a-b846-909bc5f1eac8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Objectstorage 2021-08-11T16:08:09.931Z [qtp2045766957-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "04872fed-0bae-48a4-a1a0-21e2cfc8888d" INVARIANT_UUID = "4eceec23-a04a-4e5f-92a1-b88ec970e276" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.ObjectStorage" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=04872fed-0bae-48a4-a1a0-21e2cfc8888d EntryTimestamp=2021-08-11T16:08:08.256Z InvocationID=543a2e2c-8de5-412a-b846-909bc5f1eac8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:08:09.931Z [qtp2045766957-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "04872fed-0bae-48a4-a1a0-21e2cfc8888d" INVARIANT_UUID = "4eceec23-a04a-4e5f-92a1-b88ec970e276" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.ObjectStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=04872fed-0bae-48a4-a1a0-21e2cfc8888d TargetServiceName=null EntryTimestamp=2021-08-11T16:08:08.256Z InvocationID=543a2e2c-8de5-412a-b846-909bc5f1eac8 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:08:10.145Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "04872fed-0bae-48a4-a1a0-21e2cfc8888d" INVARIANT_UUID = "4eceec23-a04a-4e5f-92a1-b88ec970e276" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.ObjectStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=04872fed-0bae-48a4-a1a0-21e2cfc8888d TargetServiceName=null EntryTimestamp=2021-08-11T16:08:08.256Z InvocationID=543a2e2c-8de5-412a-b846-909bc5f1eac8 ErrorCode=500 lock resource with id 1a22abb3-5ce1-493b-90d0-1fb0fcbc5cba 2021-08-11T16:08:10.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3be9aec2-ea99-4917-ab0a-43301fc57c5e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:08:10.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3be9aec2-ea99-4917-ab0a-43301fc57c5e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6e06ee4b 2021-08-11T16:08:10.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3be9aec2-ea99-4917-ab0a-43301fc57c5e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:08:10.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3be9aec2-ea99-4917-ab0a-43301fc57c5e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:08:11.830Z [qtp2045766957-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "04872fed-0bae-48a4-a1a0-21e2cfc8888d" INVARIANT_UUID = "4eceec23-a04a-4e5f-92a1-b88ec970e276" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "1a22abb3-5ce1-493b-90d0-1fb0fcbc5cba.assettoscatemplate" CURR_ARTIFACT_UUID = "1a22abb3-5ce1-493b-90d0-1fb0fcbc5cba.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Objectstorage-template.yml,0,1a22abb3-5ce1-493b-90d0-1fb0fcbc5cba.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=cc44769c-fded-4a52-aacc-50d20c3e8579 TargetServiceName=null EntryTimestamp=2021-08-11T16:08:08.256Z InvocationID=543a2e2c-8de5-412a-b846-909bc5f1eac8 ErrorCode=500 Starting to create Csar for component ObjectStorage 2021-08-11T16:08:12.258Z [qtp2045766957-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "04872fed-0bae-48a4-a1a0-21e2cfc8888d" INVARIANT_UUID = "4eceec23-a04a-4e5f-92a1-b88ec970e276" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "1a22abb3-5ce1-493b-90d0-1fb0fcbc5cba.assettoscatemplate" CURR_ARTIFACT_UUID = "1a22abb3-5ce1-493b-90d0-1fb0fcbc5cba.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Objectstorage-template.yml,0,1a22abb3-5ce1-493b-90d0-1fb0fcbc5cba.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=cc44769c-fded-4a52-aacc-50d20c3e8579 TargetServiceName=null EntryTimestamp=2021-08-11T16:08:08.256Z InvocationID=543a2e2c-8de5-412a-b846-909bc5f1eac8 ErrorCode=500 Ended create Csar for component ObjectStorage 2021-08-11T16:08:12.259Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "04872fed-0bae-48a4-a1a0-21e2cfc8888d" INVARIANT_UUID = "4eceec23-a04a-4e5f-92a1-b88ec970e276" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "1a22abb3-5ce1-493b-90d0-1fb0fcbc5cba.assettoscatemplate" CURR_ARTIFACT_UUID = "1a22abb3-5ce1-493b-90d0-1fb0fcbc5cba.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Objectstorage-template.yml,0,1a22abb3-5ce1-493b-90d0-1fb0fcbc5cba.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=cc44769c-fded-4a52-aacc-50d20c3e8579 TargetServiceName=null EntryTimestamp=2021-08-11T16:08:08.256Z InvocationID=543a2e2c-8de5-412a-b846-909bc5f1eac8 ErrorCode=500 lock resource with id 1a22abb3-5ce1-493b-90d0-1fb0fcbc5cba 2021-08-11T16:08:13.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=737276c2-e97e-43bc-ac98-232aff45db59 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:08:13.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=737276c2-e97e-43bc-ac98-232aff45db59 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@9e226f 2021-08-11T16:08:13.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=737276c2-e97e-43bc-ac98-232aff45db59 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:08:13.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=737276c2-e97e-43bc-ac98-232aff45db59 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:08:13.896Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "04872fed-0bae-48a4-a1a0-21e2cfc8888d" INVARIANT_UUID = "4eceec23-a04a-4e5f-92a1-b88ec970e276" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "1a22abb3-5ce1-493b-90d0-1fb0fcbc5cba.assettoscacsar" CURR_ARTIFACT_UUID = "1a22abb3-5ce1-493b-90d0-1fb0fcbc5cba.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Objectstorage-csar.csar,0,1a22abb3-5ce1-493b-90d0-1fb0fcbc5cba.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5640 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e4226dbd-41ad-40be-adf0-841b4cb9b948 TargetServiceName=null EntryTimestamp=2021-08-11T16:08:08.256Z InvocationID=543a2e2c-8de5-412a-b846-909bc5f1eac8 ErrorCode=500 LogTimestamp=2021-08-11T16:08:13.896395Z ResponseDesc=Created Exiting. 2021-08-11T16:08:13.897Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:08:13.920Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=129dc3dc-ff84-42b0-85c9-4c12b6e2591e InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:08:13.919797Z InvocationID=42cb1c53-4b58-4b35-b2ae-76b6d6dff3a7 LogTimestamp=2021-08-11T16:08:13.919877Z Entering 2021-08-11T16:08:13.920Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=129dc3dc-ff84-42b0-85c9-4c12b6e2591e InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:08:13.919797Z InvocationID=42cb1c53-4b58-4b35-b2ae-76b6d6dff3a7 LogTimestamp=2021-08-11T16:08:13.919877Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:08:14.193Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:13.921Z InvocationID=e2b035a8-a355-430a-9812-c139fb53097f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:14.193Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:13.921Z InvocationID=e2b035a8-a355-430a-9812-c139fb53097f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:14.193Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:13.921Z InvocationID=e2b035a8-a355-430a-9812-c139fb53097f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:14.193Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:13.921Z InvocationID=e2b035a8-a355-430a-9812-c139fb53097f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:14.206Z [qtp2045766957-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:13.921Z InvocationID=e2b035a8-a355-430a-9812-c139fb53097f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Blockstorage 2021-08-11T16:08:15.502Z [qtp2045766957-15] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "BlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "3ef33c8b-e4e9-45a1-9b74-dae6178f58ec" INVARIANT_UUID = "5f7e3566-19a9-48db-b4b2-85281ebf5190" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.BlockStorage" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3ef33c8b-e4e9-45a1-9b74-dae6178f58ec EntryTimestamp=2021-08-11T16:08:13.921Z InvocationID=e2b035a8-a355-430a-9812-c139fb53097f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:08:15.503Z [qtp2045766957-15] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "BlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "3ef33c8b-e4e9-45a1-9b74-dae6178f58ec" INVARIANT_UUID = "5f7e3566-19a9-48db-b4b2-85281ebf5190" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.BlockStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3ef33c8b-e4e9-45a1-9b74-dae6178f58ec TargetServiceName=null EntryTimestamp=2021-08-11T16:08:13.921Z InvocationID=e2b035a8-a355-430a-9812-c139fb53097f ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:08:15.766Z [qtp2045766957-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "BlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "3ef33c8b-e4e9-45a1-9b74-dae6178f58ec" INVARIANT_UUID = "5f7e3566-19a9-48db-b4b2-85281ebf5190" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.BlockStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3ef33c8b-e4e9-45a1-9b74-dae6178f58ec TargetServiceName=null EntryTimestamp=2021-08-11T16:08:13.921Z InvocationID=e2b035a8-a355-430a-9812-c139fb53097f ErrorCode=500 lock resource with id 8465c12b-0f5c-401b-be6c-6c5ef790fb66 2021-08-11T16:08:16.613Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d338aad-9689-4105-9ada-00eb1b160283 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:08:16.613Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d338aad-9689-4105-9ada-00eb1b160283 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@17ea8114 2021-08-11T16:08:16.613Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d338aad-9689-4105-9ada-00eb1b160283 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:08:16.613Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d338aad-9689-4105-9ada-00eb1b160283 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:08:17.381Z [qtp2045766957-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "BlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "3ef33c8b-e4e9-45a1-9b74-dae6178f58ec" INVARIANT_UUID = "5f7e3566-19a9-48db-b4b2-85281ebf5190" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "8465c12b-0f5c-401b-be6c-6c5ef790fb66.assettoscatemplate" CURR_ARTIFACT_UUID = "8465c12b-0f5c-401b-be6c-6c5ef790fb66.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Blockstorage-template.yml,0,8465c12b-0f5c-401b-be6c-6c5ef790fb66.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3907bf08-b040-4bfe-9d32-1adfc9af9c21 TargetServiceName=null EntryTimestamp=2021-08-11T16:08:13.921Z InvocationID=e2b035a8-a355-430a-9812-c139fb53097f ErrorCode=500 Starting to create Csar for component BlockStorage 2021-08-11T16:08:17.634Z [qtp2045766957-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "BlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "3ef33c8b-e4e9-45a1-9b74-dae6178f58ec" INVARIANT_UUID = "5f7e3566-19a9-48db-b4b2-85281ebf5190" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "8465c12b-0f5c-401b-be6c-6c5ef790fb66.assettoscatemplate" CURR_ARTIFACT_UUID = "8465c12b-0f5c-401b-be6c-6c5ef790fb66.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Blockstorage-template.yml,0,8465c12b-0f5c-401b-be6c-6c5ef790fb66.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3907bf08-b040-4bfe-9d32-1adfc9af9c21 TargetServiceName=null EntryTimestamp=2021-08-11T16:08:13.921Z InvocationID=e2b035a8-a355-430a-9812-c139fb53097f ErrorCode=500 Ended create Csar for component BlockStorage 2021-08-11T16:08:17.636Z [qtp2045766957-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "BlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "3ef33c8b-e4e9-45a1-9b74-dae6178f58ec" INVARIANT_UUID = "5f7e3566-19a9-48db-b4b2-85281ebf5190" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "8465c12b-0f5c-401b-be6c-6c5ef790fb66.assettoscatemplate" CURR_ARTIFACT_UUID = "8465c12b-0f5c-401b-be6c-6c5ef790fb66.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Blockstorage-template.yml,0,8465c12b-0f5c-401b-be6c-6c5ef790fb66.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3907bf08-b040-4bfe-9d32-1adfc9af9c21 TargetServiceName=null EntryTimestamp=2021-08-11T16:08:13.921Z InvocationID=e2b035a8-a355-430a-9812-c139fb53097f ErrorCode=500 lock resource with id 8465c12b-0f5c-401b-be6c-6c5ef790fb66 2021-08-11T16:08:18.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:08:18.316Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:08:19.288Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "BlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "3ef33c8b-e4e9-45a1-9b74-dae6178f58ec" INVARIANT_UUID = "5f7e3566-19a9-48db-b4b2-85281ebf5190" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "8465c12b-0f5c-401b-be6c-6c5ef790fb66.assettoscacsar" CURR_ARTIFACT_UUID = "8465c12b-0f5c-401b-be6c-6c5ef790fb66.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Blockstorage-csar.csar,0,8465c12b-0f5c-401b-be6c-6c5ef790fb66.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5366 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=5b419757-b011-4b76-9ead-b9d5b8616c36 TargetServiceName=null EntryTimestamp=2021-08-11T16:08:13.921Z InvocationID=e2b035a8-a355-430a-9812-c139fb53097f ErrorCode=500 LogTimestamp=2021-08-11T16:08:19.287532Z ResponseDesc=Created Exiting. 2021-08-11T16:08:19.288Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:08:19.311Z [qtp2045766957-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=2465cc1a-c1db-40b9-8c1f-a55f004a3bb0 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:08:19.311543Z InvocationID=6aeedf6d-4ca8-42a5-aee3-a2d6c9c441bf LogTimestamp=2021-08-11T16:08:19.311618Z Entering 2021-08-11T16:08:19.312Z [qtp2045766957-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=2465cc1a-c1db-40b9-8c1f-a55f004a3bb0 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:08:19.311543Z InvocationID=6aeedf6d-4ca8-42a5-aee3-a2d6c9c441bf LogTimestamp=2021-08-11T16:08:19.311618Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:08:19.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc839c40-1198-4f01-86e1-b1cc986f8d89 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:08:19.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc839c40-1198-4f01-86e1-b1cc986f8d89 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6ae1fb01 2021-08-11T16:08:19.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc839c40-1198-4f01-86e1-b1cc986f8d89 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:08:19.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc839c40-1198-4f01-86e1-b1cc986f8d89 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:08:19.704Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:19.313Z InvocationID=bf1c287d-5758-4037-b127-a856f2584963 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:19.704Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:19.313Z InvocationID=bf1c287d-5758-4037-b127-a856f2584963 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:19.704Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:19.313Z InvocationID=bf1c287d-5758-4037-b127-a856f2584963 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:19.704Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:19.313Z InvocationID=bf1c287d-5758-4037-b127-a856f2584963 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:19.800Z [qtp2045766957-11] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:19.313Z InvocationID=bf1c287d-5758-4037-b127-a856f2584963 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Runtime 2021-08-11T16:08:21.366Z [qtp2045766957-11] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Runtime" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "0e0e6b35-6469-4f7e-b9d2-8a6463edbe1d" INVARIANT_UUID = "e27d1034-c7a7-4816-b276-b893de810b93" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.Container.Runtime" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=0e0e6b35-6469-4f7e-b9d2-8a6463edbe1d EntryTimestamp=2021-08-11T16:08:19.313Z InvocationID=bf1c287d-5758-4037-b127-a856f2584963 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id 6f1942d0-77fe-48b7-a0dc-6382e13b86a2 2021-08-11T16:08:22.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f240dbf2-c649-4e84-8e57-b1f5b4fbb72a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:08:22.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f240dbf2-c649-4e84-8e57-b1f5b4fbb72a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@334c24a0 2021-08-11T16:08:22.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f240dbf2-c649-4e84-8e57-b1f5b4fbb72a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:08:22.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f240dbf2-c649-4e84-8e57-b1f5b4fbb72a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:08:23.006Z [qtp2045766957-11] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Runtime" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "0e0e6b35-6469-4f7e-b9d2-8a6463edbe1d" INVARIANT_UUID = "e27d1034-c7a7-4816-b276-b893de810b93" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "6f1942d0-77fe-48b7-a0dc-6382e13b86a2.assettoscatemplate" CURR_ARTIFACT_UUID = "6f1942d0-77fe-48b7-a0dc-6382e13b86a2.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Runtime-template.yml,0,6f1942d0-77fe-48b7-a0dc-6382e13b86a2.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=3693 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ee7ed780-3bc3-4192-a984-514f9c183f03 EntryTimestamp=2021-08-11T16:08:19.313Z InvocationID=bf1c287d-5758-4037-b127-a856f2584963 LogTimestamp=2021-08-11T16:08:23.006140Z ResponseDesc=Created Exiting. 2021-08-11T16:08:23.007Z [qtp2045766957-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:08:23.034Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=04fb9d3e-4a89-4f98-b3ce-12dc94b1fb12 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:08:23.034111Z InvocationID=470a6a50-d04c-4dca-898a-988efda7c7a3 LogTimestamp=2021-08-11T16:08:23.034205Z Entering 2021-08-11T16:08:23.034Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=04fb9d3e-4a89-4f98-b3ce-12dc94b1fb12 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:08:23.034111Z InvocationID=470a6a50-d04c-4dca-898a-988efda7c7a3 LogTimestamp=2021-08-11T16:08:23.034205Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:08:23.339Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:23.035Z InvocationID=86ac7e8d-938b-4bdb-a054-5d6694f87f1b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:23.339Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:23.035Z InvocationID=86ac7e8d-938b-4bdb-a054-5d6694f87f1b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:23.339Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:23.035Z InvocationID=86ac7e8d-938b-4bdb-a054-5d6694f87f1b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:23.339Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:23.035Z InvocationID=86ac7e8d-938b-4bdb-a054-5d6694f87f1b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:23.362Z [qtp2045766957-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:23.035Z InvocationID=86ac7e8d-938b-4bdb-a054-5d6694f87f1b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Application 2021-08-11T16:08:24.827Z [qtp2045766957-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Application" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "b56bdef2-cecc-4950-a358-513f5636aa10" INVARIANT_UUID = "f18e7b01-4901-4a03-ae27-9621d7201195" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.Container.Application" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b56bdef2-cecc-4950-a358-513f5636aa10 EntryTimestamp=2021-08-11T16:08:23.035Z InvocationID=86ac7e8d-938b-4bdb-a054-5d6694f87f1b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id 401a91cc-865c-4d89-a67c-b2e421fbabdf 2021-08-11T16:08:25.756Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=509f2174-e75f-44ea-911e-56cebba57888 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:08:25.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=509f2174-e75f-44ea-911e-56cebba57888 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4230f5fc 2021-08-11T16:08:25.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=509f2174-e75f-44ea-911e-56cebba57888 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:08:25.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=509f2174-e75f-44ea-911e-56cebba57888 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:08:26.487Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Application" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "b56bdef2-cecc-4950-a358-513f5636aa10" INVARIANT_UUID = "f18e7b01-4901-4a03-ae27-9621d7201195" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "401a91cc-865c-4d89-a67c-b2e421fbabdf.assettoscatemplate" CURR_ARTIFACT_UUID = "401a91cc-865c-4d89-a67c-b2e421fbabdf.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Application-template.yml,0,401a91cc-865c-4d89-a67c-b2e421fbabdf.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=3452 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=63999b82-bc71-462d-ab1d-2f9da1b40c72 EntryTimestamp=2021-08-11T16:08:23.035Z InvocationID=86ac7e8d-938b-4bdb-a054-5d6694f87f1b LogTimestamp=2021-08-11T16:08:26.487293Z ResponseDesc=Created Exiting. 2021-08-11T16:08:26.488Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:08:26.560Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=a220f12b-58e2-40fb-ba01-24e4243e7821 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:08:26.559702Z InvocationID=4e7a17aa-d7db-4917-bc06-6a477e0cf253 LogTimestamp=2021-08-11T16:08:26.559822Z Entering 2021-08-11T16:08:26.560Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=a220f12b-58e2-40fb-ba01-24e4243e7821 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:08:26.559702Z InvocationID=4e7a17aa-d7db-4917-bc06-6a477e0cf253 LogTimestamp=2021-08-11T16:08:26.559822Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:08:27.567Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:26.560Z InvocationID=ed67b9a2-9dd3-4a20-9445-788ef54adbfe PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:27.567Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:26.560Z InvocationID=ed67b9a2-9dd3-4a20-9445-788ef54adbfe PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:27.567Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:26.560Z InvocationID=ed67b9a2-9dd3-4a20-9445-788ef54adbfe PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:27.567Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:26.560Z InvocationID=ed67b9a2-9dd3-4a20-9445-788ef54adbfe PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:27.689Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:26.560Z InvocationID=ed67b9a2-9dd3-4a20-9445-788ef54adbfe PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Loadbalancer 2021-08-11T16:08:28.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c4e94dc8-958c-40ca-8f78-d7dfb0b45c6a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:08:28.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c4e94dc8-958c-40ca-8f78-d7dfb0b45c6a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1112e8f 2021-08-11T16:08:28.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c4e94dc8-958c-40ca-8f78-d7dfb0b45c6a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:08:28.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c4e94dc8-958c-40ca-8f78-d7dfb0b45c6a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:08:29.784Z [qtp2045766957-12] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "LoadBalancer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "f378e9d0-b9cb-4e0f-9fda-4d11a374163c" INVARIANT_UUID = "ab0790b8-9894-4119-909f-fe01a42e0353" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.LoadBalancer" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f378e9d0-b9cb-4e0f-9fda-4d11a374163c EntryTimestamp=2021-08-11T16:08:26.560Z InvocationID=ed67b9a2-9dd3-4a20-9445-788ef54adbfe PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:08:29.785Z [qtp2045766957-12] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "LoadBalancer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "f378e9d0-b9cb-4e0f-9fda-4d11a374163c" INVARIANT_UUID = "ab0790b8-9894-4119-909f-fe01a42e0353" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.LoadBalancer" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f378e9d0-b9cb-4e0f-9fda-4d11a374163c TargetServiceName=null EntryTimestamp=2021-08-11T16:08:26.560Z InvocationID=ed67b9a2-9dd3-4a20-9445-788ef54adbfe ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:08:30.215Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "LoadBalancer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "f378e9d0-b9cb-4e0f-9fda-4d11a374163c" INVARIANT_UUID = "ab0790b8-9894-4119-909f-fe01a42e0353" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.LoadBalancer" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f378e9d0-b9cb-4e0f-9fda-4d11a374163c TargetServiceName=null EntryTimestamp=2021-08-11T16:08:26.560Z InvocationID=ed67b9a2-9dd3-4a20-9445-788ef54adbfe ErrorCode=500 lock resource with id 45b24d16-1898-4fee-aaee-8cab741108d4 2021-08-11T16:08:31.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5d171df6-984d-4b6a-abe1-aa255f8ad2f4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:08:31.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5d171df6-984d-4b6a-abe1-aa255f8ad2f4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@64cde45f 2021-08-11T16:08:31.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5d171df6-984d-4b6a-abe1-aa255f8ad2f4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:08:31.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5d171df6-984d-4b6a-abe1-aa255f8ad2f4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:08:31.822Z [qtp2045766957-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "LoadBalancer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "f378e9d0-b9cb-4e0f-9fda-4d11a374163c" INVARIANT_UUID = "ab0790b8-9894-4119-909f-fe01a42e0353" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "45b24d16-1898-4fee-aaee-8cab741108d4.assettoscatemplate" CURR_ARTIFACT_UUID = "45b24d16-1898-4fee-aaee-8cab741108d4.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Loadbalancer-template.yml,0,45b24d16-1898-4fee-aaee-8cab741108d4.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1a68515a-7837-48ee-9190-dca9eb66333d TargetServiceName=null EntryTimestamp=2021-08-11T16:08:26.560Z InvocationID=ed67b9a2-9dd3-4a20-9445-788ef54adbfe ErrorCode=500 Starting to create Csar for component LoadBalancer 2021-08-11T16:08:32.044Z [qtp2045766957-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "LoadBalancer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "f378e9d0-b9cb-4e0f-9fda-4d11a374163c" INVARIANT_UUID = "ab0790b8-9894-4119-909f-fe01a42e0353" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "45b24d16-1898-4fee-aaee-8cab741108d4.assettoscatemplate" CURR_ARTIFACT_UUID = "45b24d16-1898-4fee-aaee-8cab741108d4.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Loadbalancer-template.yml,0,45b24d16-1898-4fee-aaee-8cab741108d4.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1a68515a-7837-48ee-9190-dca9eb66333d TargetServiceName=null EntryTimestamp=2021-08-11T16:08:26.560Z InvocationID=ed67b9a2-9dd3-4a20-9445-788ef54adbfe ErrorCode=500 Ended create Csar for component LoadBalancer 2021-08-11T16:08:32.045Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "LoadBalancer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "f378e9d0-b9cb-4e0f-9fda-4d11a374163c" INVARIANT_UUID = "ab0790b8-9894-4119-909f-fe01a42e0353" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "45b24d16-1898-4fee-aaee-8cab741108d4.assettoscatemplate" CURR_ARTIFACT_UUID = "45b24d16-1898-4fee-aaee-8cab741108d4.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Loadbalancer-template.yml,0,45b24d16-1898-4fee-aaee-8cab741108d4.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1a68515a-7837-48ee-9190-dca9eb66333d TargetServiceName=null EntryTimestamp=2021-08-11T16:08:26.560Z InvocationID=ed67b9a2-9dd3-4a20-9445-788ef54adbfe ErrorCode=500 lock resource with id 45b24d16-1898-4fee-aaee-8cab741108d4 2021-08-11T16:08:33.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:08:33.322Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:08:33.694Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "LoadBalancer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "f378e9d0-b9cb-4e0f-9fda-4d11a374163c" INVARIANT_UUID = "ab0790b8-9894-4119-909f-fe01a42e0353" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "45b24d16-1898-4fee-aaee-8cab741108d4.assettoscacsar" CURR_ARTIFACT_UUID = "45b24d16-1898-4fee-aaee-8cab741108d4.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Loadbalancer-csar.csar,0,45b24d16-1898-4fee-aaee-8cab741108d4.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=7134 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b15f39cb-cbb4-48ce-b562-1dace90410d7 TargetServiceName=null EntryTimestamp=2021-08-11T16:08:26.560Z InvocationID=ed67b9a2-9dd3-4a20-9445-788ef54adbfe ErrorCode=500 LogTimestamp=2021-08-11T16:08:33.694127Z ResponseDesc=Created Exiting. 2021-08-11T16:08:33.695Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:08:33.726Z [qtp2045766957-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=2405f2f2-3222-45e0-b9de-16c62eae9100 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:08:33.726037Z InvocationID=bc07d7a0-7a8d-40b6-ae4c-005d04d6ac2b LogTimestamp=2021-08-11T16:08:33.726121Z Entering 2021-08-11T16:08:33.726Z [qtp2045766957-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=2405f2f2-3222-45e0-b9de-16c62eae9100 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:08:33.726037Z InvocationID=bc07d7a0-7a8d-40b6-ae4c-005d04d6ac2b LogTimestamp=2021-08-11T16:08:33.726121Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:08:34.030Z [qtp2045766957-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:33.727Z InvocationID=3671fa7e-6b41-4d6e-8ea2-5d142f4e4852 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:34.030Z [qtp2045766957-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:33.727Z InvocationID=3671fa7e-6b41-4d6e-8ea2-5d142f4e4852 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:34.030Z [qtp2045766957-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:33.727Z InvocationID=3671fa7e-6b41-4d6e-8ea2-5d142f4e4852 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:34.030Z [qtp2045766957-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:33.727Z InvocationID=3671fa7e-6b41-4d6e-8ea2-5d142f4e4852 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:34.057Z [qtp2045766957-20] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:33.727Z InvocationID=3671fa7e-6b41-4d6e-8ea2-5d142f4e4852 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Port 2021-08-11T16:08:34.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5138dd0a-04e0-4a2e-a494-f00120b2264a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:08:34.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5138dd0a-04e0-4a2e-a494-f00120b2264a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@36f6b542 2021-08-11T16:08:34.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5138dd0a-04e0-4a2e-a494-f00120b2264a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:08:34.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5138dd0a-04e0-4a2e-a494-f00120b2264a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:08:35.487Z [qtp2045766957-20] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Port" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "49f9b411-a221-49e8-beff-b62985206cd6" INVARIANT_UUID = "f954a4d4-ff4b-46a1-aae2-60172ca1e8a7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.network.Port" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=49f9b411-a221-49e8-beff-b62985206cd6 EntryTimestamp=2021-08-11T16:08:33.727Z InvocationID=3671fa7e-6b41-4d6e-8ea2-5d142f4e4852 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:08:35.487Z [qtp2045766957-20] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Port" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "49f9b411-a221-49e8-beff-b62985206cd6" INVARIANT_UUID = "f954a4d4-ff4b-46a1-aae2-60172ca1e8a7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.network.Port" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=49f9b411-a221-49e8-beff-b62985206cd6 TargetServiceName=null EntryTimestamp=2021-08-11T16:08:33.727Z InvocationID=3671fa7e-6b41-4d6e-8ea2-5d142f4e4852 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:08:35.704Z [qtp2045766957-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Port" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "49f9b411-a221-49e8-beff-b62985206cd6" INVARIANT_UUID = "f954a4d4-ff4b-46a1-aae2-60172ca1e8a7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.network.Port" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=49f9b411-a221-49e8-beff-b62985206cd6 TargetServiceName=null EntryTimestamp=2021-08-11T16:08:33.727Z InvocationID=3671fa7e-6b41-4d6e-8ea2-5d142f4e4852 ErrorCode=500 lock resource with id 32654392-90db-4108-89ca-aafa2fb175a5 2021-08-11T16:08:37.343Z [qtp2045766957-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Port" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "49f9b411-a221-49e8-beff-b62985206cd6" INVARIANT_UUID = "f954a4d4-ff4b-46a1-aae2-60172ca1e8a7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "32654392-90db-4108-89ca-aafa2fb175a5.assettoscatemplate" CURR_ARTIFACT_UUID = "32654392-90db-4108-89ca-aafa2fb175a5.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Port-template.yml,0,32654392-90db-4108-89ca-aafa2fb175a5.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=73796fdf-1032-47f5-9de4-adb649f00270 TargetServiceName=null EntryTimestamp=2021-08-11T16:08:33.727Z InvocationID=3671fa7e-6b41-4d6e-8ea2-5d142f4e4852 ErrorCode=500 Starting to create Csar for component Port 2021-08-11T16:08:37.557Z [qtp2045766957-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Port" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "49f9b411-a221-49e8-beff-b62985206cd6" INVARIANT_UUID = "f954a4d4-ff4b-46a1-aae2-60172ca1e8a7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "32654392-90db-4108-89ca-aafa2fb175a5.assettoscatemplate" CURR_ARTIFACT_UUID = "32654392-90db-4108-89ca-aafa2fb175a5.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Port-template.yml,0,32654392-90db-4108-89ca-aafa2fb175a5.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=73796fdf-1032-47f5-9de4-adb649f00270 TargetServiceName=null EntryTimestamp=2021-08-11T16:08:33.727Z InvocationID=3671fa7e-6b41-4d6e-8ea2-5d142f4e4852 ErrorCode=500 Ended create Csar for component Port 2021-08-11T16:08:37.557Z [qtp2045766957-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Port" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "49f9b411-a221-49e8-beff-b62985206cd6" INVARIANT_UUID = "f954a4d4-ff4b-46a1-aae2-60172ca1e8a7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "32654392-90db-4108-89ca-aafa2fb175a5.assettoscatemplate" CURR_ARTIFACT_UUID = "32654392-90db-4108-89ca-aafa2fb175a5.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Port-template.yml,0,32654392-90db-4108-89ca-aafa2fb175a5.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=73796fdf-1032-47f5-9de4-adb649f00270 TargetServiceName=null EntryTimestamp=2021-08-11T16:08:33.727Z InvocationID=3671fa7e-6b41-4d6e-8ea2-5d142f4e4852 ErrorCode=500 lock resource with id 32654392-90db-4108-89ca-aafa2fb175a5 2021-08-11T16:08:37.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a4d05d2-0568-4c6d-8fb6-7054b44f1caf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:08:37.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a4d05d2-0568-4c6d-8fb6-7054b44f1caf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@732c2975 2021-08-11T16:08:37.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a4d05d2-0568-4c6d-8fb6-7054b44f1caf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:08:37.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a4d05d2-0568-4c6d-8fb6-7054b44f1caf PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:08:39.195Z [qtp2045766957-20] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Port" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "49f9b411-a221-49e8-beff-b62985206cd6" INVARIANT_UUID = "f954a4d4-ff4b-46a1-aae2-60172ca1e8a7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "32654392-90db-4108-89ca-aafa2fb175a5.assettoscacsar" CURR_ARTIFACT_UUID = "32654392-90db-4108-89ca-aafa2fb175a5.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Port-csar.csar,0,32654392-90db-4108-89ca-aafa2fb175a5.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5467 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c5997e69-4532-403d-aa06-8035beb5178d TargetServiceName=null EntryTimestamp=2021-08-11T16:08:33.727Z InvocationID=3671fa7e-6b41-4d6e-8ea2-5d142f4e4852 ErrorCode=500 LogTimestamp=2021-08-11T16:08:39.194678Z ResponseDesc=Created Exiting. 2021-08-11T16:08:39.195Z [qtp2045766957-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:08:39.255Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=250703ef-4d70-4e34-9223-8edabc463c09 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:08:39.254838Z InvocationID=1417992d-f2c9-4c4c-963b-9fc51091654a LogTimestamp=2021-08-11T16:08:39.254986Z Entering 2021-08-11T16:08:39.255Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=250703ef-4d70-4e34-9223-8edabc463c09 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:08:39.254838Z InvocationID=1417992d-f2c9-4c4c-963b-9fc51091654a LogTimestamp=2021-08-11T16:08:39.254986Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:08:39.626Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:39.256Z InvocationID=28e818ab-a79c-4125-a9d7-701db57984f0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:39.626Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:39.256Z InvocationID=28e818ab-a79c-4125-a9d7-701db57984f0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:39.626Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:39.256Z InvocationID=28e818ab-a79c-4125-a9d7-701db57984f0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:39.626Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:39.256Z InvocationID=28e818ab-a79c-4125-a9d7-701db57984f0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:39.637Z [qtp2045766957-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:39.256Z InvocationID=28e818ab-a79c-4125-a9d7-701db57984f0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Network 2021-08-11T16:08:40.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fbdf00d2-13ae-42d3-986a-3040d41fb42b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:08:40.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fbdf00d2-13ae-42d3-986a-3040d41fb42b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7bc104c9 2021-08-11T16:08:40.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fbdf00d2-13ae-42d3-986a-3040d41fb42b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:08:40.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fbdf00d2-13ae-42d3-986a-3040d41fb42b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:08:41.058Z [qtp2045766957-15] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Network" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "d2301011-050f-4581-aa1b-8792b58f15e3" INVARIANT_UUID = "3a9575b8-2d8c-4969-ba5a-2a370968fde4" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.network.Network" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d2301011-050f-4581-aa1b-8792b58f15e3 EntryTimestamp=2021-08-11T16:08:39.256Z InvocationID=28e818ab-a79c-4125-a9d7-701db57984f0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:08:41.059Z [qtp2045766957-15] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Network" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "d2301011-050f-4581-aa1b-8792b58f15e3" INVARIANT_UUID = "3a9575b8-2d8c-4969-ba5a-2a370968fde4" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.network.Network" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d2301011-050f-4581-aa1b-8792b58f15e3 TargetServiceName=null EntryTimestamp=2021-08-11T16:08:39.256Z InvocationID=28e818ab-a79c-4125-a9d7-701db57984f0 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:08:41.264Z [qtp2045766957-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Network" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "d2301011-050f-4581-aa1b-8792b58f15e3" INVARIANT_UUID = "3a9575b8-2d8c-4969-ba5a-2a370968fde4" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.network.Network" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d2301011-050f-4581-aa1b-8792b58f15e3 TargetServiceName=null EntryTimestamp=2021-08-11T16:08:39.256Z InvocationID=28e818ab-a79c-4125-a9d7-701db57984f0 ErrorCode=500 lock resource with id c4c56679-b561-4a97-9df0-62412e10a43c 2021-08-11T16:08:42.875Z [qtp2045766957-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Network" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "d2301011-050f-4581-aa1b-8792b58f15e3" INVARIANT_UUID = "3a9575b8-2d8c-4969-ba5a-2a370968fde4" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c4c56679-b561-4a97-9df0-62412e10a43c.assettoscatemplate" CURR_ARTIFACT_UUID = "c4c56679-b561-4a97-9df0-62412e10a43c.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Network-template.yml,0,c4c56679-b561-4a97-9df0-62412e10a43c.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=23e48872-5665-4a34-906f-1d97d8461588 TargetServiceName=null EntryTimestamp=2021-08-11T16:08:39.256Z InvocationID=28e818ab-a79c-4125-a9d7-701db57984f0 ErrorCode=500 Starting to create Csar for component Network 2021-08-11T16:08:43.081Z [qtp2045766957-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Network" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "d2301011-050f-4581-aa1b-8792b58f15e3" INVARIANT_UUID = "3a9575b8-2d8c-4969-ba5a-2a370968fde4" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c4c56679-b561-4a97-9df0-62412e10a43c.assettoscatemplate" CURR_ARTIFACT_UUID = "c4c56679-b561-4a97-9df0-62412e10a43c.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Network-template.yml,0,c4c56679-b561-4a97-9df0-62412e10a43c.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=23e48872-5665-4a34-906f-1d97d8461588 TargetServiceName=null EntryTimestamp=2021-08-11T16:08:39.256Z InvocationID=28e818ab-a79c-4125-a9d7-701db57984f0 ErrorCode=500 Ended create Csar for component Network 2021-08-11T16:08:43.082Z [qtp2045766957-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Network" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "d2301011-050f-4581-aa1b-8792b58f15e3" INVARIANT_UUID = "3a9575b8-2d8c-4969-ba5a-2a370968fde4" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c4c56679-b561-4a97-9df0-62412e10a43c.assettoscatemplate" CURR_ARTIFACT_UUID = "c4c56679-b561-4a97-9df0-62412e10a43c.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Network-template.yml,0,c4c56679-b561-4a97-9df0-62412e10a43c.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=23e48872-5665-4a34-906f-1d97d8461588 TargetServiceName=null EntryTimestamp=2021-08-11T16:08:39.256Z InvocationID=28e818ab-a79c-4125-a9d7-701db57984f0 ErrorCode=500 lock resource with id c4c56679-b561-4a97-9df0-62412e10a43c 2021-08-11T16:08:43.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c169f60-5c8b-47c9-a04b-a40b0b76241f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:08:43.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c169f60-5c8b-47c9-a04b-a40b0b76241f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@70a5f8f2 2021-08-11T16:08:43.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c169f60-5c8b-47c9-a04b-a40b0b76241f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:08:43.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c169f60-5c8b-47c9-a04b-a40b0b76241f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:08:44.742Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Network" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "d2301011-050f-4581-aa1b-8792b58f15e3" INVARIANT_UUID = "3a9575b8-2d8c-4969-ba5a-2a370968fde4" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c4c56679-b561-4a97-9df0-62412e10a43c.assettoscacsar" CURR_ARTIFACT_UUID = "c4c56679-b561-4a97-9df0-62412e10a43c.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Network-csar.csar,0,c4c56679-b561-4a97-9df0-62412e10a43c.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5485 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=0e4243d9-b1cc-4d48-946d-642899f401ae TargetServiceName=null EntryTimestamp=2021-08-11T16:08:39.256Z InvocationID=28e818ab-a79c-4125-a9d7-701db57984f0 ErrorCode=500 LogTimestamp=2021-08-11T16:08:44.741925Z ResponseDesc=Created Exiting. 2021-08-11T16:08:44.742Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:08:44.763Z [qtp2045766957-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=7976491b-1f68-4ea2-9c9a-77362a6541c7 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:08:44.763419Z InvocationID=2b78c5fe-0200-4557-8a25-750ecfde5c34 LogTimestamp=2021-08-11T16:08:44.763480Z Entering 2021-08-11T16:08:44.763Z [qtp2045766957-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=7976491b-1f68-4ea2-9c9a-77362a6541c7 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:08:44.763419Z InvocationID=2b78c5fe-0200-4557-8a25-750ecfde5c34 LogTimestamp=2021-08-11T16:08:44.763480Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:08:45.130Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:44.764Z InvocationID=4befc3b4-c01b-480b-85d9-05350600c1ed PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:45.131Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:44.764Z InvocationID=4befc3b4-c01b-480b-85d9-05350600c1ed PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:45.131Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:44.764Z InvocationID=4befc3b4-c01b-480b-85d9-05350600c1ed PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:45.131Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:44.764Z InvocationID=4befc3b4-c01b-480b-85d9-05350600c1ed PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:45.163Z [qtp2045766957-11] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:44.764Z InvocationID=4befc3b4-c01b-480b-85d9-05350600c1ed PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Globalnetwork 2021-08-11T16:08:46.785Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18bc131c-a5bb-41e3-a73f-c0817959db58 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:08:46.786Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18bc131c-a5bb-41e3-a73f-c0817959db58 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@73fa1f01 2021-08-11T16:08:46.786Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18bc131c-a5bb-41e3-a73f-c0817959db58 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:08:46.786Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18bc131c-a5bb-41e3-a73f-c0817959db58 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:08:46.961Z [qtp2045766957-11] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "GlobalNetwork" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "d6d177bb-9d39-4d59-ac8b-cd2681468fd7" INVARIANT_UUID = "8d871a56-c62d-4407-8cbb-8a3aafeeeeea" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.nodes.network.Network" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d6d177bb-9d39-4d59-ac8b-cd2681468fd7 EntryTimestamp=2021-08-11T16:08:44.764Z InvocationID=4befc3b4-c01b-480b-85d9-05350600c1ed PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id 52cc663d-2b8b-4cab-80a0-1feea2c5bcc4 2021-08-11T16:08:48.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:08:48.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:08:48.677Z [qtp2045766957-11] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "GlobalNetwork" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "d6d177bb-9d39-4d59-ac8b-cd2681468fd7" INVARIANT_UUID = "8d871a56-c62d-4407-8cbb-8a3aafeeeeea" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "52cc663d-2b8b-4cab-80a0-1feea2c5bcc4.assettoscatemplate" CURR_ARTIFACT_UUID = "52cc663d-2b8b-4cab-80a0-1feea2c5bcc4.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Globalnetwork-template.yml,0,52cc663d-2b8b-4cab-80a0-1feea2c5bcc4.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=3912 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=efb428de-e4cf-4a05-8ff6-a3a76e2ec464 EntryTimestamp=2021-08-11T16:08:44.764Z InvocationID=4befc3b4-c01b-480b-85d9-05350600c1ed LogTimestamp=2021-08-11T16:08:48.676636Z ResponseDesc=Created Exiting. 2021-08-11T16:08:48.677Z [qtp2045766957-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:08:48.750Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=e863767d-2fe6-4f4d-8751-b48d560fe96c InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:08:48.750363Z InvocationID=42489b10-4fab-4321-bcbe-d7ff84dc5a18 LogTimestamp=2021-08-11T16:08:48.750419Z Entering 2021-08-11T16:08:48.750Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=e863767d-2fe6-4f4d-8751-b48d560fe96c InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:08:48.750363Z InvocationID=42489b10-4fab-4321-bcbe-d7ff84dc5a18 LogTimestamp=2021-08-11T16:08:48.750419Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:08:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:08:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:08:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:08:50.549Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=390e260a-42fa-47ff-9769-fb3a0e4848fe PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:08:50.549Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=390e260a-42fa-47ff-9769-fb3a0e4848fe PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@610fe78a 2021-08-11T16:08:50.549Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=390e260a-42fa-47ff-9769-fb3a0e4848fe PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:08:50.549Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=390e260a-42fa-47ff-9769-fb3a0e4848fe PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:08:51.348Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:48.750Z InvocationID=7c000be5-cc1c-41e6-bd0e-242ad8d516eb PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:51.348Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:48.750Z InvocationID=7c000be5-cc1c-41e6-bd0e-242ad8d516eb PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:51.348Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:48.750Z InvocationID=7c000be5-cc1c-41e6-bd0e-242ad8d516eb PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:51.348Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:48.750Z InvocationID=7c000be5-cc1c-41e6-bd0e-242ad8d516eb PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:08:51.649Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:08:48.750Z InvocationID=7c000be5-cc1c-41e6-bd0e-242ad8d516eb PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Globalport 2021-08-11T16:08:53.960Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7211a36e-915e-498a-b654-9315e7278bc1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:08:53.960Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7211a36e-915e-498a-b654-9315e7278bc1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@129b402e 2021-08-11T16:08:53.961Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7211a36e-915e-498a-b654-9315e7278bc1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:08:53.961Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7211a36e-915e-498a-b654-9315e7278bc1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:08:56.874Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc04087c-4f91-4f57-8131-e3aaee9e8943 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:08:56.874Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc04087c-4f91-4f57-8131-e3aaee9e8943 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4d2648e7 2021-08-11T16:08:56.874Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc04087c-4f91-4f57-8131-e3aaee9e8943 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:08:56.874Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc04087c-4f91-4f57-8131-e3aaee9e8943 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:08:57.759Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "GlobalPort" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "acb29356-7751-487a-ab6a-d864f17688f2" INVARIANT_UUID = "a259b3c0-f9a3-4fef-804f-1ca33b7a143d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.network.Port" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=acb29356-7751-487a-ab6a-d864f17688f2 EntryTimestamp=2021-08-11T16:08:48.750Z InvocationID=7c000be5-cc1c-41e6-bd0e-242ad8d516eb PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id 5f2bc1ca-7c4f-4371-b9ea-faf88dbbe76e 2021-08-11T16:08:59.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e373285-8240-4859-aaf7-5ef03b454bba PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:08:59.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e373285-8240-4859-aaf7-5ef03b454bba PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6e074455 2021-08-11T16:08:59.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e373285-8240-4859-aaf7-5ef03b454bba PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:08:59.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e373285-8240-4859-aaf7-5ef03b454bba PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:09:00.220Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "GlobalPort" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "acb29356-7751-487a-ab6a-d864f17688f2" INVARIANT_UUID = "a259b3c0-f9a3-4fef-804f-1ca33b7a143d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "5f2bc1ca-7c4f-4371-b9ea-faf88dbbe76e.assettoscatemplate" CURR_ARTIFACT_UUID = "5f2bc1ca-7c4f-4371-b9ea-faf88dbbe76e.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Globalport-template.yml,0,5f2bc1ca-7c4f-4371-b9ea-faf88dbbe76e.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=11469 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=6addf9b5-4f9e-459c-a632-58bfd456fc8d EntryTimestamp=2021-08-11T16:08:48.750Z InvocationID=7c000be5-cc1c-41e6-bd0e-242ad8d516eb LogTimestamp=2021-08-11T16:09:00.219631Z ResponseDesc=Created Exiting. 2021-08-11T16:09:00.220Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:09:00.273Z [qtp2045766957-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=0fee0eec-c3b6-4659-99a5-8375b73d8a03 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:09:00.273103Z InvocationID=001fd9a7-f95d-4063-b72b-486c719a5743 LogTimestamp=2021-08-11T16:09:00.273177Z Entering 2021-08-11T16:09:00.275Z [qtp2045766957-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=0fee0eec-c3b6-4659-99a5-8375b73d8a03 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:09:00.273103Z InvocationID=001fd9a7-f95d-4063-b72b-486c719a5743 LogTimestamp=2021-08-11T16:09:00.273177Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:09:02.065Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:00.275Z InvocationID=66ac4ca8-8db6-4c3d-89f8-04bc4b068076 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:02.066Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:00.275Z InvocationID=66ac4ca8-8db6-4c3d-89f8-04bc4b068076 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:02.066Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:00.275Z InvocationID=66ac4ca8-8db6-4c3d-89f8-04bc4b068076 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:02.066Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:00.275Z InvocationID=66ac4ca8-8db6-4c3d-89f8-04bc4b068076 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:02.183Z [qtp2045766957-11] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:00.275Z InvocationID=66ac4ca8-8db6-4c3d-89f8-04bc4b068076 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Globalcompute 2021-08-11T16:09:02.989Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9fe1c767-dcc1-4d61-819a-5c2ce6bc8be1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:09:02.989Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9fe1c767-dcc1-4d61-819a-5c2ce6bc8be1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@702adda8 2021-08-11T16:09:02.989Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9fe1c767-dcc1-4d61-819a-5c2ce6bc8be1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:09:02.989Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9fe1c767-dcc1-4d61-819a-5c2ce6bc8be1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:09:03.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:09:03.328Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:09:04.256Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:09:05.460Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d263a68f-23f3-4353-9b0b-4566e8c7b6cb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:09:05.461Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d263a68f-23f3-4353-9b0b-4566e8c7b6cb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@387869b3 2021-08-11T16:09:05.461Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d263a68f-23f3-4353-9b0b-4566e8c7b6cb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:09:05.461Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d263a68f-23f3-4353-9b0b-4566e8c7b6cb PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:09:05.793Z [qtp2045766957-11] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "GlobalCompute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "f447cc03-1227-4f01-919e-129e8ce59ee2" INVARIANT_UUID = "ff761c7e-e908-41ef-abe5-a8d8317a0b7a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.nodes.Compute" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f447cc03-1227-4f01-919e-129e8ce59ee2 EntryTimestamp=2021-08-11T16:09:00.275Z InvocationID=66ac4ca8-8db6-4c3d-89f8-04bc4b068076 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id 72de9412-69b4-4327-b77d-acd097cba8af 2021-08-11T16:09:07.488Z [qtp2045766957-11] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "GlobalCompute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "f447cc03-1227-4f01-919e-129e8ce59ee2" INVARIANT_UUID = "ff761c7e-e908-41ef-abe5-a8d8317a0b7a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "72de9412-69b4-4327-b77d-acd097cba8af.assettoscatemplate" CURR_ARTIFACT_UUID = "72de9412-69b4-4327-b77d-acd097cba8af.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Globalcompute-template.yml,0,72de9412-69b4-4327-b77d-acd097cba8af.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=7213 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=8397472c-ac22-4f52-9ffd-17a6c77e0a9b EntryTimestamp=2021-08-11T16:09:00.275Z InvocationID=66ac4ca8-8db6-4c3d-89f8-04bc4b068076 LogTimestamp=2021-08-11T16:09:07.488498Z ResponseDesc=Created Exiting. 2021-08-11T16:09:07.489Z [qtp2045766957-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:09:07.510Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=a3b0e9ba-8ca0-45cb-891d-b89c5a1f370b InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:09:07.509741Z InvocationID=560fe340-848b-4c92-8628-71d0c36c9f8b LogTimestamp=2021-08-11T16:09:07.509818Z Entering 2021-08-11T16:09:07.510Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=a3b0e9ba-8ca0-45cb-891d-b89c5a1f370b InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:09:07.509741Z InvocationID=560fe340-848b-4c92-8628-71d0c36c9f8b LogTimestamp=2021-08-11T16:09:07.509818Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:09:07.782Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cdcda092-118d-44d4-b47d-cbaaddcd12ba PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:09:07.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cdcda092-118d-44d4-b47d-cbaaddcd12ba PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@47e67f2b 2021-08-11T16:09:07.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cdcda092-118d-44d4-b47d-cbaaddcd12ba PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:09:07.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cdcda092-118d-44d4-b47d-cbaaddcd12ba PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:09:08.164Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:07.510Z InvocationID=7409fed2-6cbd-4685-a54a-b0b69e7536a8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:08.164Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:07.510Z InvocationID=7409fed2-6cbd-4685-a54a-b0b69e7536a8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:08.164Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:07.510Z InvocationID=7409fed2-6cbd-4685-a54a-b0b69e7536a8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:08.164Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:07.510Z InvocationID=7409fed2-6cbd-4685-a54a-b0b69e7536a8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:08.164Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:07.510Z InvocationID=7409fed2-6cbd-4685-a54a-b0b69e7536a8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Volume 2021-08-11T16:09:09.457Z [qtp2045766957-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "volume" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "3bd0c9a8-46a4-4e68-a401-757c7b62b34a" INVARIANT_UUID = "e931075c-95e6-479b-b50d-7f3bb00737f8" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.nodes.volume" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3bd0c9a8-46a4-4e68-a401-757c7b62b34a EntryTimestamp=2021-08-11T16:09:07.510Z InvocationID=7409fed2-6cbd-4685-a54a-b0b69e7536a8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:09:09.476Z [qtp2045766957-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "volume" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "3bd0c9a8-46a4-4e68-a401-757c7b62b34a" INVARIANT_UUID = "e931075c-95e6-479b-b50d-7f3bb00737f8" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.nodes.volume" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3bd0c9a8-46a4-4e68-a401-757c7b62b34a TargetServiceName=null EntryTimestamp=2021-08-11T16:09:07.510Z InvocationID=7409fed2-6cbd-4685-a54a-b0b69e7536a8 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:09:09.697Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "volume" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "3bd0c9a8-46a4-4e68-a401-757c7b62b34a" INVARIANT_UUID = "e931075c-95e6-479b-b50d-7f3bb00737f8" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.nodes.volume" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3bd0c9a8-46a4-4e68-a401-757c7b62b34a TargetServiceName=null EntryTimestamp=2021-08-11T16:09:07.510Z InvocationID=7409fed2-6cbd-4685-a54a-b0b69e7536a8 ErrorCode=500 lock resource with id ed26c2c6-f35c-4992-8b13-54828e87e3fb 2021-08-11T16:09:10.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ca742de-b51f-4577-ae38-69acbcde0d23 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:09:10.627Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ca742de-b51f-4577-ae38-69acbcde0d23 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1317ae6e 2021-08-11T16:09:10.627Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ca742de-b51f-4577-ae38-69acbcde0d23 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:09:10.627Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ca742de-b51f-4577-ae38-69acbcde0d23 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:09:11.334Z [qtp2045766957-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "volume" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "3bd0c9a8-46a4-4e68-a401-757c7b62b34a" INVARIANT_UUID = "e931075c-95e6-479b-b50d-7f3bb00737f8" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "ed26c2c6-f35c-4992-8b13-54828e87e3fb.assettoscatemplate" CURR_ARTIFACT_UUID = "ed26c2c6-f35c-4992-8b13-54828e87e3fb.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Volume-template.yml,0,ed26c2c6-f35c-4992-8b13-54828e87e3fb.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7918db7d-3ee4-462e-be9f-9004edf5f101 TargetServiceName=null EntryTimestamp=2021-08-11T16:09:07.510Z InvocationID=7409fed2-6cbd-4685-a54a-b0b69e7536a8 ErrorCode=500 Starting to create Csar for component volume 2021-08-11T16:09:11.640Z [qtp2045766957-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "volume" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "3bd0c9a8-46a4-4e68-a401-757c7b62b34a" INVARIANT_UUID = "e931075c-95e6-479b-b50d-7f3bb00737f8" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "ed26c2c6-f35c-4992-8b13-54828e87e3fb.assettoscatemplate" CURR_ARTIFACT_UUID = "ed26c2c6-f35c-4992-8b13-54828e87e3fb.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Volume-template.yml,0,ed26c2c6-f35c-4992-8b13-54828e87e3fb.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7918db7d-3ee4-462e-be9f-9004edf5f101 TargetServiceName=null EntryTimestamp=2021-08-11T16:09:07.510Z InvocationID=7409fed2-6cbd-4685-a54a-b0b69e7536a8 ErrorCode=500 Ended create Csar for component volume 2021-08-11T16:09:11.640Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "volume" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "3bd0c9a8-46a4-4e68-a401-757c7b62b34a" INVARIANT_UUID = "e931075c-95e6-479b-b50d-7f3bb00737f8" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "ed26c2c6-f35c-4992-8b13-54828e87e3fb.assettoscatemplate" CURR_ARTIFACT_UUID = "ed26c2c6-f35c-4992-8b13-54828e87e3fb.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Volume-template.yml,0,ed26c2c6-f35c-4992-8b13-54828e87e3fb.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7918db7d-3ee4-462e-be9f-9004edf5f101 TargetServiceName=null EntryTimestamp=2021-08-11T16:09:07.510Z InvocationID=7409fed2-6cbd-4685-a54a-b0b69e7536a8 ErrorCode=500 lock resource with id ed26c2c6-f35c-4992-8b13-54828e87e3fb 2021-08-11T16:09:13.281Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "volume" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "3bd0c9a8-46a4-4e68-a401-757c7b62b34a" INVARIANT_UUID = "e931075c-95e6-479b-b50d-7f3bb00737f8" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "ed26c2c6-f35c-4992-8b13-54828e87e3fb.assettoscacsar" CURR_ARTIFACT_UUID = "ed26c2c6-f35c-4992-8b13-54828e87e3fb.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Volume-csar.csar,0,ed26c2c6-f35c-4992-8b13-54828e87e3fb.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5771 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=087073d3-4d4f-419c-a647-ed86411724fc TargetServiceName=null EntryTimestamp=2021-08-11T16:09:07.510Z InvocationID=7409fed2-6cbd-4685-a54a-b0b69e7536a8 ErrorCode=500 LogTimestamp=2021-08-11T16:09:13.281136Z ResponseDesc=Created Exiting. 2021-08-11T16:09:13.282Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:09:13.304Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=4420d52c-676a-4957-b712-cff21546e936 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:09:13.303641Z InvocationID=865fcf27-cea4-4964-aed1-dd4babf51cb7 LogTimestamp=2021-08-11T16:09:13.303735Z Entering 2021-08-11T16:09:13.304Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=4420d52c-676a-4957-b712-cff21546e936 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:09:13.303641Z InvocationID=865fcf27-cea4-4964-aed1-dd4babf51cb7 LogTimestamp=2021-08-11T16:09:13.303735Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:09:13.759Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:13.304Z InvocationID=c67c5adb-2694-47f1-846b-224f8f5ce149 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:13.759Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:13.304Z InvocationID=c67c5adb-2694-47f1-846b-224f8f5ce149 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:13.759Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:13.304Z InvocationID=c67c5adb-2694-47f1-846b-224f8f5ce149 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:13.759Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:13.304Z InvocationID=c67c5adb-2694-47f1-846b-224f8f5ce149 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:13.759Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:13.304Z InvocationID=c67c5adb-2694-47f1-846b-224f8f5ce149 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Cindervolume 2021-08-11T16:09:13.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2aa15f02-71b5-47b9-8d3e-663cd89aca98 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:09:13.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2aa15f02-71b5-47b9-8d3e-663cd89aca98 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3576f639 2021-08-11T16:09:13.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2aa15f02-71b5-47b9-8d3e-663cd89aca98 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:09:13.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2aa15f02-71b5-47b9-8d3e-663cd89aca98 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:09:15.067Z [qtp2045766957-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "CinderVolume" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "fbb9acb1-bbfa-4b45-a037-f85683b51978" INVARIANT_UUID = "3322dd57-0db0-4044-a156-c2990c7c85c1" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.nodes.heat.cinder.Volume" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=fbb9acb1-bbfa-4b45-a037-f85683b51978 EntryTimestamp=2021-08-11T16:09:13.304Z InvocationID=c67c5adb-2694-47f1-846b-224f8f5ce149 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:09:15.067Z [qtp2045766957-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "CinderVolume" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "fbb9acb1-bbfa-4b45-a037-f85683b51978" INVARIANT_UUID = "3322dd57-0db0-4044-a156-c2990c7c85c1" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.nodes.heat.cinder.Volume" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=fbb9acb1-bbfa-4b45-a037-f85683b51978 TargetServiceName=null EntryTimestamp=2021-08-11T16:09:13.304Z InvocationID=c67c5adb-2694-47f1-846b-224f8f5ce149 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:09:15.397Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "CinderVolume" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "fbb9acb1-bbfa-4b45-a037-f85683b51978" INVARIANT_UUID = "3322dd57-0db0-4044-a156-c2990c7c85c1" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.nodes.heat.cinder.Volume" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=fbb9acb1-bbfa-4b45-a037-f85683b51978 TargetServiceName=null EntryTimestamp=2021-08-11T16:09:13.304Z InvocationID=c67c5adb-2694-47f1-846b-224f8f5ce149 ErrorCode=500 lock resource with id 7effa02c-9248-42c1-bf7f-d4d365762a37 2021-08-11T16:09:16.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a2e8026a-9913-4405-ae7e-17855a9fd037 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:09:16.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a2e8026a-9913-4405-ae7e-17855a9fd037 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4eb824d5 2021-08-11T16:09:16.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a2e8026a-9913-4405-ae7e-17855a9fd037 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:09:16.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a2e8026a-9913-4405-ae7e-17855a9fd037 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:09:17.008Z [qtp2045766957-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "CinderVolume" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "fbb9acb1-bbfa-4b45-a037-f85683b51978" INVARIANT_UUID = "3322dd57-0db0-4044-a156-c2990c7c85c1" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "7effa02c-9248-42c1-bf7f-d4d365762a37.assettoscatemplate" CURR_ARTIFACT_UUID = "7effa02c-9248-42c1-bf7f-d4d365762a37.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Cindervolume-template.yml,0,7effa02c-9248-42c1-bf7f-d4d365762a37.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ac239f5c-d03d-46fe-a155-7d49fda3c3ce TargetServiceName=null EntryTimestamp=2021-08-11T16:09:13.304Z InvocationID=c67c5adb-2694-47f1-846b-224f8f5ce149 ErrorCode=500 Starting to create Csar for component CinderVolume 2021-08-11T16:09:17.211Z [qtp2045766957-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "CinderVolume" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "fbb9acb1-bbfa-4b45-a037-f85683b51978" INVARIANT_UUID = "3322dd57-0db0-4044-a156-c2990c7c85c1" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "7effa02c-9248-42c1-bf7f-d4d365762a37.assettoscatemplate" CURR_ARTIFACT_UUID = "7effa02c-9248-42c1-bf7f-d4d365762a37.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Cindervolume-template.yml,0,7effa02c-9248-42c1-bf7f-d4d365762a37.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ac239f5c-d03d-46fe-a155-7d49fda3c3ce TargetServiceName=null EntryTimestamp=2021-08-11T16:09:13.304Z InvocationID=c67c5adb-2694-47f1-846b-224f8f5ce149 ErrorCode=500 Ended create Csar for component CinderVolume 2021-08-11T16:09:17.212Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "CinderVolume" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "fbb9acb1-bbfa-4b45-a037-f85683b51978" INVARIANT_UUID = "3322dd57-0db0-4044-a156-c2990c7c85c1" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "7effa02c-9248-42c1-bf7f-d4d365762a37.assettoscatemplate" CURR_ARTIFACT_UUID = "7effa02c-9248-42c1-bf7f-d4d365762a37.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Cindervolume-template.yml,0,7effa02c-9248-42c1-bf7f-d4d365762a37.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ac239f5c-d03d-46fe-a155-7d49fda3c3ce TargetServiceName=null EntryTimestamp=2021-08-11T16:09:13.304Z InvocationID=c67c5adb-2694-47f1-846b-224f8f5ce149 ErrorCode=500 lock resource with id 7effa02c-9248-42c1-bf7f-d4d365762a37 2021-08-11T16:09:18.305Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:09:18.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:09:18.855Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "CinderVolume" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "fbb9acb1-bbfa-4b45-a037-f85683b51978" INVARIANT_UUID = "3322dd57-0db0-4044-a156-c2990c7c85c1" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "7effa02c-9248-42c1-bf7f-d4d365762a37.assettoscacsar" CURR_ARTIFACT_UUID = "7effa02c-9248-42c1-bf7f-d4d365762a37.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Cindervolume-csar.csar,0,7effa02c-9248-42c1-bf7f-d4d365762a37.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5551 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1db88a97-7959-4d38-a778-75741fd257bf TargetServiceName=null EntryTimestamp=2021-08-11T16:09:13.304Z InvocationID=c67c5adb-2694-47f1-846b-224f8f5ce149 ErrorCode=500 LogTimestamp=2021-08-11T16:09:18.855517Z ResponseDesc=Created Exiting. 2021-08-11T16:09:18.856Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:09:18.878Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=83c299e1-9acd-4666-b553-a349ccb4e389 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:09:18.878015Z InvocationID=e37e87e4-ba04-491f-b7f0-a5ac9cde631b LogTimestamp=2021-08-11T16:09:18.878089Z Entering 2021-08-11T16:09:18.878Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=83c299e1-9acd-4666-b553-a349ccb4e389 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:09:18.878015Z InvocationID=e37e87e4-ba04-491f-b7f0-a5ac9cde631b LogTimestamp=2021-08-11T16:09:18.878089Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:09:19.246Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:18.878Z InvocationID=c69d90cd-33e3-4f58-a3f7-6170fa19b2ed PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:19.246Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:18.878Z InvocationID=c69d90cd-33e3-4f58-a3f7-6170fa19b2ed PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:19.246Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:18.878Z InvocationID=c69d90cd-33e3-4f58-a3f7-6170fa19b2ed PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:19.246Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:18.878Z InvocationID=c69d90cd-33e3-4f58-a3f7-6170fa19b2ed PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:19.259Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:18.878Z InvocationID=c69d90cd-33e3-4f58-a3f7-6170fa19b2ed PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Contrailvirtualnetwork 2021-08-11T16:09:19.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3179ade8-269e-4e25-8d6c-071e3c33f4f9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:09:19.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3179ade8-269e-4e25-8d6c-071e3c33f4f9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3a53150e 2021-08-11T16:09:19.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3179ade8-269e-4e25-8d6c-071e3c33f4f9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:09:19.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3179ade8-269e-4e25-8d6c-071e3c33f4f9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:09:20.597Z [qtp2045766957-18] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailVirtualNetwork" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "d5ef7ae3-a2ef-4813-89f7-afaa4676c108" INVARIANT_UUID = "fb3a5a44-231a-4b43-865f-217538ea13a7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.nodes.heat.network.contrail.VirtualNetwork" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d5ef7ae3-a2ef-4813-89f7-afaa4676c108 EntryTimestamp=2021-08-11T16:09:18.878Z InvocationID=c69d90cd-33e3-4f58-a3f7-6170fa19b2ed PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:09:20.597Z [qtp2045766957-18] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailVirtualNetwork" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "d5ef7ae3-a2ef-4813-89f7-afaa4676c108" INVARIANT_UUID = "fb3a5a44-231a-4b43-865f-217538ea13a7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.nodes.heat.network.contrail.VirtualNetwork" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d5ef7ae3-a2ef-4813-89f7-afaa4676c108 TargetServiceName=null EntryTimestamp=2021-08-11T16:09:18.878Z InvocationID=c69d90cd-33e3-4f58-a3f7-6170fa19b2ed ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:09:20.786Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailVirtualNetwork" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "d5ef7ae3-a2ef-4813-89f7-afaa4676c108" INVARIANT_UUID = "fb3a5a44-231a-4b43-865f-217538ea13a7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.nodes.heat.network.contrail.VirtualNetwork" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d5ef7ae3-a2ef-4813-89f7-afaa4676c108 TargetServiceName=null EntryTimestamp=2021-08-11T16:09:18.878Z InvocationID=c69d90cd-33e3-4f58-a3f7-6170fa19b2ed ErrorCode=500 lock resource with id 71d15130-7a38-45d0-b79b-f10877ff59c0 2021-08-11T16:09:22.431Z [qtp2045766957-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailVirtualNetwork" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "d5ef7ae3-a2ef-4813-89f7-afaa4676c108" INVARIANT_UUID = "fb3a5a44-231a-4b43-865f-217538ea13a7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "71d15130-7a38-45d0-b79b-f10877ff59c0.assettoscatemplate" CURR_ARTIFACT_UUID = "71d15130-7a38-45d0-b79b-f10877ff59c0.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailvirtualnetwork-template.yml,0,71d15130-7a38-45d0-b79b-f10877ff59c0.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a0cb03cf-74f6-4fe5-81c1-f8936fe76d7b TargetServiceName=null EntryTimestamp=2021-08-11T16:09:18.878Z InvocationID=c69d90cd-33e3-4f58-a3f7-6170fa19b2ed ErrorCode=500 Starting to create Csar for component ContrailVirtualNetwork 2021-08-11T16:09:22.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=145ec430-849d-4227-af8a-b1373ecee60f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:09:22.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=145ec430-849d-4227-af8a-b1373ecee60f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1499b8f4 2021-08-11T16:09:22.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=145ec430-849d-4227-af8a-b1373ecee60f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:09:22.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=145ec430-849d-4227-af8a-b1373ecee60f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:09:22.626Z [qtp2045766957-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailVirtualNetwork" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "d5ef7ae3-a2ef-4813-89f7-afaa4676c108" INVARIANT_UUID = "fb3a5a44-231a-4b43-865f-217538ea13a7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "71d15130-7a38-45d0-b79b-f10877ff59c0.assettoscatemplate" CURR_ARTIFACT_UUID = "71d15130-7a38-45d0-b79b-f10877ff59c0.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailvirtualnetwork-template.yml,0,71d15130-7a38-45d0-b79b-f10877ff59c0.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a0cb03cf-74f6-4fe5-81c1-f8936fe76d7b TargetServiceName=null EntryTimestamp=2021-08-11T16:09:18.878Z InvocationID=c69d90cd-33e3-4f58-a3f7-6170fa19b2ed ErrorCode=500 Ended create Csar for component ContrailVirtualNetwork 2021-08-11T16:09:22.626Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailVirtualNetwork" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "d5ef7ae3-a2ef-4813-89f7-afaa4676c108" INVARIANT_UUID = "fb3a5a44-231a-4b43-865f-217538ea13a7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "71d15130-7a38-45d0-b79b-f10877ff59c0.assettoscatemplate" CURR_ARTIFACT_UUID = "71d15130-7a38-45d0-b79b-f10877ff59c0.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailvirtualnetwork-template.yml,0,71d15130-7a38-45d0-b79b-f10877ff59c0.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a0cb03cf-74f6-4fe5-81c1-f8936fe76d7b TargetServiceName=null EntryTimestamp=2021-08-11T16:09:18.878Z InvocationID=c69d90cd-33e3-4f58-a3f7-6170fa19b2ed ErrorCode=500 lock resource with id 71d15130-7a38-45d0-b79b-f10877ff59c0 2021-08-11T16:09:24.264Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailVirtualNetwork" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "d5ef7ae3-a2ef-4813-89f7-afaa4676c108" INVARIANT_UUID = "fb3a5a44-231a-4b43-865f-217538ea13a7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "71d15130-7a38-45d0-b79b-f10877ff59c0.assettoscacsar" CURR_ARTIFACT_UUID = "71d15130-7a38-45d0-b79b-f10877ff59c0.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Contrailvirtualnetwork-csar.csar,0,71d15130-7a38-45d0-b79b-f10877ff59c0.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5385 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1c74f1cd-3de6-4352-851b-f72d473a3c21 TargetServiceName=null EntryTimestamp=2021-08-11T16:09:18.878Z InvocationID=c69d90cd-33e3-4f58-a3f7-6170fa19b2ed ErrorCode=500 LogTimestamp=2021-08-11T16:09:24.263986Z ResponseDesc=Created Exiting. 2021-08-11T16:09:24.264Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:09:24.294Z [qtp2045766957-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=5ff53dc5-3005-430d-8ca9-dd39283198e6 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:09:24.293913Z InvocationID=18f29ab2-0acb-466e-90cb-41441d843034 LogTimestamp=2021-08-11T16:09:24.293988Z Entering 2021-08-11T16:09:24.294Z [qtp2045766957-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=5ff53dc5-3005-430d-8ca9-dd39283198e6 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:09:24.293913Z InvocationID=18f29ab2-0acb-466e-90cb-41441d843034 LogTimestamp=2021-08-11T16:09:24.293988Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:09:24.577Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:24.295Z InvocationID=afc0e339-44ad-49d6-bc26-8907bff2dafe PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:24.577Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:24.295Z InvocationID=afc0e339-44ad-49d6-bc26-8907bff2dafe PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:24.577Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:24.295Z InvocationID=afc0e339-44ad-49d6-bc26-8907bff2dafe PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:24.577Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:24.295Z InvocationID=afc0e339-44ad-49d6-bc26-8907bff2dafe PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:24.589Z [qtp2045766957-11] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:24.295Z InvocationID=afc0e339-44ad-49d6-bc26-8907bff2dafe PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Neutronnet 2021-08-11T16:09:25.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2bf0bf13-0873-4a8e-8078-d93172a14029 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:09:25.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2bf0bf13-0873-4a8e-8078-d93172a14029 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@165c4360 2021-08-11T16:09:25.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2bf0bf13-0873-4a8e-8078-d93172a14029 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:09:25.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2bf0bf13-0873-4a8e-8078-d93172a14029 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:09:25.877Z [qtp2045766957-11] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NeutronNet" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "2010a25e-3afc-46c7-add7-0f72a8e3d403" INVARIANT_UUID = "c9a37dcb-fd87-41be-bed7-a555ec03fe45" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.nodes.heat.network.neutron.Net" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2010a25e-3afc-46c7-add7-0f72a8e3d403 EntryTimestamp=2021-08-11T16:09:24.295Z InvocationID=afc0e339-44ad-49d6-bc26-8907bff2dafe PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:09:25.877Z [qtp2045766957-11] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NeutronNet" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "2010a25e-3afc-46c7-add7-0f72a8e3d403" INVARIANT_UUID = "c9a37dcb-fd87-41be-bed7-a555ec03fe45" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.nodes.heat.network.neutron.Net" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2010a25e-3afc-46c7-add7-0f72a8e3d403 TargetServiceName=null EntryTimestamp=2021-08-11T16:09:24.295Z InvocationID=afc0e339-44ad-49d6-bc26-8907bff2dafe ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:09:26.452Z [qtp2045766957-11] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NeutronNet" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "2010a25e-3afc-46c7-add7-0f72a8e3d403" INVARIANT_UUID = "c9a37dcb-fd87-41be-bed7-a555ec03fe45" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.nodes.heat.network.neutron.Net" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2010a25e-3afc-46c7-add7-0f72a8e3d403 TargetServiceName=null EntryTimestamp=2021-08-11T16:09:24.295Z InvocationID=afc0e339-44ad-49d6-bc26-8907bff2dafe ErrorCode=500 lock resource with id 1e1e0c31-01f8-4cd4-bb66-342235b10f58 2021-08-11T16:09:28.260Z [qtp2045766957-11] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NeutronNet" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "2010a25e-3afc-46c7-add7-0f72a8e3d403" INVARIANT_UUID = "c9a37dcb-fd87-41be-bed7-a555ec03fe45" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "1e1e0c31-01f8-4cd4-bb66-342235b10f58.assettoscatemplate" CURR_ARTIFACT_UUID = "1e1e0c31-01f8-4cd4-bb66-342235b10f58.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Neutronnet-template.yml,0,1e1e0c31-01f8-4cd4-bb66-342235b10f58.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ea85eb2c-8959-4312-8f16-cb03e72cd13f TargetServiceName=null EntryTimestamp=2021-08-11T16:09:24.295Z InvocationID=afc0e339-44ad-49d6-bc26-8907bff2dafe ErrorCode=500 Starting to create Csar for component NeutronNet 2021-08-11T16:09:28.402Z [qtp2045766957-11] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NeutronNet" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "2010a25e-3afc-46c7-add7-0f72a8e3d403" INVARIANT_UUID = "c9a37dcb-fd87-41be-bed7-a555ec03fe45" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "1e1e0c31-01f8-4cd4-bb66-342235b10f58.assettoscatemplate" CURR_ARTIFACT_UUID = "1e1e0c31-01f8-4cd4-bb66-342235b10f58.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Neutronnet-template.yml,0,1e1e0c31-01f8-4cd4-bb66-342235b10f58.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ea85eb2c-8959-4312-8f16-cb03e72cd13f TargetServiceName=null EntryTimestamp=2021-08-11T16:09:24.295Z InvocationID=afc0e339-44ad-49d6-bc26-8907bff2dafe ErrorCode=500 Ended create Csar for component NeutronNet 2021-08-11T16:09:28.402Z [qtp2045766957-11] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NeutronNet" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "2010a25e-3afc-46c7-add7-0f72a8e3d403" INVARIANT_UUID = "c9a37dcb-fd87-41be-bed7-a555ec03fe45" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "1e1e0c31-01f8-4cd4-bb66-342235b10f58.assettoscatemplate" CURR_ARTIFACT_UUID = "1e1e0c31-01f8-4cd4-bb66-342235b10f58.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Neutronnet-template.yml,0,1e1e0c31-01f8-4cd4-bb66-342235b10f58.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ea85eb2c-8959-4312-8f16-cb03e72cd13f TargetServiceName=null EntryTimestamp=2021-08-11T16:09:24.295Z InvocationID=afc0e339-44ad-49d6-bc26-8907bff2dafe ErrorCode=500 lock resource with id 1e1e0c31-01f8-4cd4-bb66-342235b10f58 2021-08-11T16:09:28.852Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=976dbfe8-4d03-45da-b621-b6d0454c5d6d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:09:28.852Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=976dbfe8-4d03-45da-b621-b6d0454c5d6d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@194d53f2 2021-08-11T16:09:28.852Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=976dbfe8-4d03-45da-b621-b6d0454c5d6d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:09:28.852Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=976dbfe8-4d03-45da-b621-b6d0454c5d6d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:09:30.277Z [qtp2045766957-11] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NeutronNet" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "2010a25e-3afc-46c7-add7-0f72a8e3d403" INVARIANT_UUID = "c9a37dcb-fd87-41be-bed7-a555ec03fe45" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "1e1e0c31-01f8-4cd4-bb66-342235b10f58.assettoscacsar" CURR_ARTIFACT_UUID = "1e1e0c31-01f8-4cd4-bb66-342235b10f58.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Neutronnet-csar.csar,0,1e1e0c31-01f8-4cd4-bb66-342235b10f58.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5982 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=129820cf-87b9-49cf-95fb-390624c45d54 TargetServiceName=null EntryTimestamp=2021-08-11T16:09:24.295Z InvocationID=afc0e339-44ad-49d6-bc26-8907bff2dafe ErrorCode=500 LogTimestamp=2021-08-11T16:09:30.277134Z ResponseDesc=Created Exiting. 2021-08-11T16:09:30.277Z [qtp2045766957-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:09:30.296Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=331be12c-a835-4bb2-87d8-3176dae64c23 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:09:30.295889Z InvocationID=5b7a36af-3153-42d5-aae0-536694f3f887 LogTimestamp=2021-08-11T16:09:30.295966Z Entering 2021-08-11T16:09:30.296Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=331be12c-a835-4bb2-87d8-3176dae64c23 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:09:30.295889Z InvocationID=5b7a36af-3153-42d5-aae0-536694f3f887 LogTimestamp=2021-08-11T16:09:30.295966Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:09:30.981Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:30.296Z InvocationID=c0f6db35-f3cd-4eeb-a13a-b3995502cf76 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:30.982Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:30.296Z InvocationID=c0f6db35-f3cd-4eeb-a13a-b3995502cf76 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:30.982Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:30.296Z InvocationID=c0f6db35-f3cd-4eeb-a13a-b3995502cf76 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:30.982Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:30.296Z InvocationID=c0f6db35-f3cd-4eeb-a13a-b3995502cf76 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:31.051Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:30.296Z InvocationID=c0f6db35-f3cd-4eeb-a13a-b3995502cf76 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Neutronport 2021-08-11T16:09:31.701Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bbead120-f6a6-4826-bf51-2ad7ba18309b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:09:31.701Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bbead120-f6a6-4826-bf51-2ad7ba18309b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3b94deb0 2021-08-11T16:09:31.702Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bbead120-f6a6-4826-bf51-2ad7ba18309b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:09:31.702Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bbead120-f6a6-4826-bf51-2ad7ba18309b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:09:32.575Z [qtp2045766957-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NeutronPort" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "42cf1bb5-6724-45a2-aa1a-4c7632d0eeaa" INVARIANT_UUID = "1120aa5d-99b4-44ea-b529-96f65bf8d155" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.heat.network.neutron.Port" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=42cf1bb5-6724-45a2-aa1a-4c7632d0eeaa EntryTimestamp=2021-08-11T16:09:30.296Z InvocationID=c0f6db35-f3cd-4eeb-a13a-b3995502cf76 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:09:32.576Z [qtp2045766957-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NeutronPort" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "42cf1bb5-6724-45a2-aa1a-4c7632d0eeaa" INVARIANT_UUID = "1120aa5d-99b4-44ea-b529-96f65bf8d155" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.heat.network.neutron.Port" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=42cf1bb5-6724-45a2-aa1a-4c7632d0eeaa TargetServiceName=null EntryTimestamp=2021-08-11T16:09:30.296Z InvocationID=c0f6db35-f3cd-4eeb-a13a-b3995502cf76 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:09:32.865Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NeutronPort" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "42cf1bb5-6724-45a2-aa1a-4c7632d0eeaa" INVARIANT_UUID = "1120aa5d-99b4-44ea-b529-96f65bf8d155" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.heat.network.neutron.Port" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=42cf1bb5-6724-45a2-aa1a-4c7632d0eeaa TargetServiceName=null EntryTimestamp=2021-08-11T16:09:30.296Z InvocationID=c0f6db35-f3cd-4eeb-a13a-b3995502cf76 ErrorCode=500 lock resource with id 21200ef2-42de-452c-85ec-e8a581659360 2021-08-11T16:09:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:09:33.317Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:09:34.475Z [qtp2045766957-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NeutronPort" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "42cf1bb5-6724-45a2-aa1a-4c7632d0eeaa" INVARIANT_UUID = "1120aa5d-99b4-44ea-b529-96f65bf8d155" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "21200ef2-42de-452c-85ec-e8a581659360.assettoscatemplate" CURR_ARTIFACT_UUID = "21200ef2-42de-452c-85ec-e8a581659360.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Neutronport-template.yml,0,21200ef2-42de-452c-85ec-e8a581659360.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=49ab4939-9643-4ab1-8034-1c3472c9e20d TargetServiceName=null EntryTimestamp=2021-08-11T16:09:30.296Z InvocationID=c0f6db35-f3cd-4eeb-a13a-b3995502cf76 ErrorCode=500 Starting to create Csar for component NeutronPort 2021-08-11T16:09:34.627Z [qtp2045766957-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NeutronPort" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "42cf1bb5-6724-45a2-aa1a-4c7632d0eeaa" INVARIANT_UUID = "1120aa5d-99b4-44ea-b529-96f65bf8d155" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "21200ef2-42de-452c-85ec-e8a581659360.assettoscatemplate" CURR_ARTIFACT_UUID = "21200ef2-42de-452c-85ec-e8a581659360.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Neutronport-template.yml,0,21200ef2-42de-452c-85ec-e8a581659360.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=49ab4939-9643-4ab1-8034-1c3472c9e20d TargetServiceName=null EntryTimestamp=2021-08-11T16:09:30.296Z InvocationID=c0f6db35-f3cd-4eeb-a13a-b3995502cf76 ErrorCode=500 Ended create Csar for component NeutronPort 2021-08-11T16:09:34.628Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NeutronPort" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "42cf1bb5-6724-45a2-aa1a-4c7632d0eeaa" INVARIANT_UUID = "1120aa5d-99b4-44ea-b529-96f65bf8d155" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "21200ef2-42de-452c-85ec-e8a581659360.assettoscatemplate" CURR_ARTIFACT_UUID = "21200ef2-42de-452c-85ec-e8a581659360.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Neutronport-template.yml,0,21200ef2-42de-452c-85ec-e8a581659360.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=49ab4939-9643-4ab1-8034-1c3472c9e20d TargetServiceName=null EntryTimestamp=2021-08-11T16:09:30.296Z InvocationID=c0f6db35-f3cd-4eeb-a13a-b3995502cf76 ErrorCode=500 lock resource with id 21200ef2-42de-452c-85ec-e8a581659360 2021-08-11T16:09:34.693Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=328c4134-b630-4c66-a2bf-8f6d34dec158 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:09:34.693Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=328c4134-b630-4c66-a2bf-8f6d34dec158 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@62138f2e 2021-08-11T16:09:34.693Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=328c4134-b630-4c66-a2bf-8f6d34dec158 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:09:34.693Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=328c4134-b630-4c66-a2bf-8f6d34dec158 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:09:36.253Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NeutronPort" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "42cf1bb5-6724-45a2-aa1a-4c7632d0eeaa" INVARIANT_UUID = "1120aa5d-99b4-44ea-b529-96f65bf8d155" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "21200ef2-42de-452c-85ec-e8a581659360.assettoscacsar" CURR_ARTIFACT_UUID = "21200ef2-42de-452c-85ec-e8a581659360.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Neutronport-csar.csar,0,21200ef2-42de-452c-85ec-e8a581659360.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5957 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=4c3f2001-5f8f-4ff3-9a6f-bcc9c61c8098 TargetServiceName=null EntryTimestamp=2021-08-11T16:09:30.296Z InvocationID=c0f6db35-f3cd-4eeb-a13a-b3995502cf76 ErrorCode=500 LogTimestamp=2021-08-11T16:09:36.253454Z ResponseDesc=Created Exiting. 2021-08-11T16:09:36.254Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:09:36.275Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=dd764ca0-b435-4da2-99f3-06aaa70b5b84 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:09:36.274759Z InvocationID=d0a7c381-1c7d-4b3a-a533-9a0829c114ff LogTimestamp=2021-08-11T16:09:36.274841Z Entering 2021-08-11T16:09:36.275Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=dd764ca0-b435-4da2-99f3-06aaa70b5b84 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:09:36.274759Z InvocationID=d0a7c381-1c7d-4b3a-a533-9a0829c114ff LogTimestamp=2021-08-11T16:09:36.274841Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:09:36.553Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:36.275Z InvocationID=e498161e-9177-40f6-affa-2045b7d9b68d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:36.553Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:36.275Z InvocationID=e498161e-9177-40f6-affa-2045b7d9b68d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:36.553Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:36.275Z InvocationID=e498161e-9177-40f6-affa-2045b7d9b68d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:36.553Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:36.275Z InvocationID=e498161e-9177-40f6-affa-2045b7d9b68d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:36.553Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:36.275Z InvocationID=e498161e-9177-40f6-affa-2045b7d9b68d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Novaserver 2021-08-11T16:09:37.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c81fc9f-de1f-4a75-8082-b215cd5ca475 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:09:37.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c81fc9f-de1f-4a75-8082-b215cd5ca475 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@16a0874f 2021-08-11T16:09:37.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c81fc9f-de1f-4a75-8082-b215cd5ca475 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:09:37.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c81fc9f-de1f-4a75-8082-b215cd5ca475 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:09:37.958Z [qtp2045766957-14] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NovaServer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "4db3e80c-a504-4063-a45d-7ecc620a7229" INVARIANT_UUID = "2f568108-dc78-4449-bc38-0d36263a1cfe" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.nodes.heat.nova.Server" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=4db3e80c-a504-4063-a45d-7ecc620a7229 EntryTimestamp=2021-08-11T16:09:36.275Z InvocationID=e498161e-9177-40f6-affa-2045b7d9b68d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:09:37.958Z [qtp2045766957-14] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NovaServer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "4db3e80c-a504-4063-a45d-7ecc620a7229" INVARIANT_UUID = "2f568108-dc78-4449-bc38-0d36263a1cfe" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.nodes.heat.nova.Server" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=4db3e80c-a504-4063-a45d-7ecc620a7229 TargetServiceName=null EntryTimestamp=2021-08-11T16:09:36.275Z InvocationID=e498161e-9177-40f6-affa-2045b7d9b68d ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:09:38.755Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NovaServer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "4db3e80c-a504-4063-a45d-7ecc620a7229" INVARIANT_UUID = "2f568108-dc78-4449-bc38-0d36263a1cfe" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.nodes.heat.nova.Server" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=4db3e80c-a504-4063-a45d-7ecc620a7229 TargetServiceName=null EntryTimestamp=2021-08-11T16:09:36.275Z InvocationID=e498161e-9177-40f6-affa-2045b7d9b68d ErrorCode=500 lock resource with id 87220d3f-aa1f-41a3-99b1-15296ca49795 2021-08-11T16:09:40.685Z [qtp2045766957-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NovaServer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "4db3e80c-a504-4063-a45d-7ecc620a7229" INVARIANT_UUID = "2f568108-dc78-4449-bc38-0d36263a1cfe" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "87220d3f-aa1f-41a3-99b1-15296ca49795.assettoscatemplate" CURR_ARTIFACT_UUID = "87220d3f-aa1f-41a3-99b1-15296ca49795.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Novaserver-template.yml,0,87220d3f-aa1f-41a3-99b1-15296ca49795.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ab2c38c8-30e8-4fe9-a878-08a2f25d3b60 TargetServiceName=null EntryTimestamp=2021-08-11T16:09:36.275Z InvocationID=e498161e-9177-40f6-affa-2045b7d9b68d ErrorCode=500 Starting to create Csar for component NovaServer 2021-08-11T16:09:40.834Z [qtp2045766957-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NovaServer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "4db3e80c-a504-4063-a45d-7ecc620a7229" INVARIANT_UUID = "2f568108-dc78-4449-bc38-0d36263a1cfe" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "87220d3f-aa1f-41a3-99b1-15296ca49795.assettoscatemplate" CURR_ARTIFACT_UUID = "87220d3f-aa1f-41a3-99b1-15296ca49795.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Novaserver-template.yml,0,87220d3f-aa1f-41a3-99b1-15296ca49795.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ab2c38c8-30e8-4fe9-a878-08a2f25d3b60 TargetServiceName=null EntryTimestamp=2021-08-11T16:09:36.275Z InvocationID=e498161e-9177-40f6-affa-2045b7d9b68d ErrorCode=500 Ended create Csar for component NovaServer 2021-08-11T16:09:40.834Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NovaServer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "4db3e80c-a504-4063-a45d-7ecc620a7229" INVARIANT_UUID = "2f568108-dc78-4449-bc38-0d36263a1cfe" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "87220d3f-aa1f-41a3-99b1-15296ca49795.assettoscatemplate" CURR_ARTIFACT_UUID = "87220d3f-aa1f-41a3-99b1-15296ca49795.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Novaserver-template.yml,0,87220d3f-aa1f-41a3-99b1-15296ca49795.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ab2c38c8-30e8-4fe9-a878-08a2f25d3b60 TargetServiceName=null EntryTimestamp=2021-08-11T16:09:36.275Z InvocationID=e498161e-9177-40f6-affa-2045b7d9b68d ErrorCode=500 lock resource with id 87220d3f-aa1f-41a3-99b1-15296ca49795 2021-08-11T16:09:40.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5d42dd25-9578-49c7-a76b-904674a542ca PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:09:40.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5d42dd25-9578-49c7-a76b-904674a542ca PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@658184aa 2021-08-11T16:09:40.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5d42dd25-9578-49c7-a76b-904674a542ca PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:09:40.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5d42dd25-9578-49c7-a76b-904674a542ca PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:09:42.512Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NovaServer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "4db3e80c-a504-4063-a45d-7ecc620a7229" INVARIANT_UUID = "2f568108-dc78-4449-bc38-0d36263a1cfe" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "87220d3f-aa1f-41a3-99b1-15296ca49795.assettoscacsar" CURR_ARTIFACT_UUID = "87220d3f-aa1f-41a3-99b1-15296ca49795.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Novaserver-csar.csar,0,87220d3f-aa1f-41a3-99b1-15296ca49795.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=6236 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=6e34b931-20d4-4ef7-8391-3e188053a8be TargetServiceName=null EntryTimestamp=2021-08-11T16:09:36.275Z InvocationID=e498161e-9177-40f6-affa-2045b7d9b68d ErrorCode=500 LogTimestamp=2021-08-11T16:09:42.511974Z ResponseDesc=Created Exiting. 2021-08-11T16:09:42.512Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:09:42.531Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=da810c9c-cce2-4402-beb7-9f219e675e23 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:09:42.531177Z InvocationID=565fcfe5-bb92-453f-8f9b-3aeec3cff88c LogTimestamp=2021-08-11T16:09:42.531237Z Entering 2021-08-11T16:09:42.531Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=da810c9c-cce2-4402-beb7-9f219e675e23 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:09:42.531177Z InvocationID=565fcfe5-bb92-453f-8f9b-3aeec3cff88c LogTimestamp=2021-08-11T16:09:42.531237Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:09:42.936Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:42.531Z InvocationID=6fac35d8-893a-4ca0-984f-abd7d3537cbb PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:42.936Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:42.531Z InvocationID=6fac35d8-893a-4ca0-984f-abd7d3537cbb PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:42.936Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:42.531Z InvocationID=6fac35d8-893a-4ca0-984f-abd7d3537cbb PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:42.936Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:42.531Z InvocationID=6fac35d8-893a-4ca0-984f-abd7d3537cbb PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:42.947Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:42.531Z InvocationID=6fac35d8-893a-4ca0-984f-abd7d3537cbb PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Extvl 2021-08-11T16:09:43.765Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4348b3d0-fb01-4dc5-9029-49aa1bb2a6bd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:09:43.765Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4348b3d0-fb01-4dc5-9029-49aa1bb2a6bd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@78fdb7f4 2021-08-11T16:09:43.765Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4348b3d0-fb01-4dc5-9029-49aa1bb2a6bd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:09:43.766Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4348b3d0-fb01-4dc5-9029-49aa1bb2a6bd PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:09:44.285Z [qtp2045766957-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ExtVL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "abfc0f07-aea8-4edb-9cb6-68a85a9d3ca8" INVARIANT_UUID = "a72ce58a-d31d-4aa7-88f8-8c9c09ab9101" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.extVL" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=abfc0f07-aea8-4edb-9cb6-68a85a9d3ca8 EntryTimestamp=2021-08-11T16:09:42.531Z InvocationID=6fac35d8-893a-4ca0-984f-abd7d3537cbb PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:09:44.285Z [qtp2045766957-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ExtVL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "abfc0f07-aea8-4edb-9cb6-68a85a9d3ca8" INVARIANT_UUID = "a72ce58a-d31d-4aa7-88f8-8c9c09ab9101" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.extVL" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=abfc0f07-aea8-4edb-9cb6-68a85a9d3ca8 TargetServiceName=null EntryTimestamp=2021-08-11T16:09:42.531Z InvocationID=6fac35d8-893a-4ca0-984f-abd7d3537cbb ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:09:44.524Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ExtVL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "abfc0f07-aea8-4edb-9cb6-68a85a9d3ca8" INVARIANT_UUID = "a72ce58a-d31d-4aa7-88f8-8c9c09ab9101" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.extVL" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=abfc0f07-aea8-4edb-9cb6-68a85a9d3ca8 TargetServiceName=null EntryTimestamp=2021-08-11T16:09:42.531Z InvocationID=6fac35d8-893a-4ca0-984f-abd7d3537cbb ErrorCode=500 lock resource with id fe8d3770-835a-4b7b-b977-65d8042f20fa 2021-08-11T16:09:46.124Z [qtp2045766957-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ExtVL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "abfc0f07-aea8-4edb-9cb6-68a85a9d3ca8" INVARIANT_UUID = "a72ce58a-d31d-4aa7-88f8-8c9c09ab9101" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "fe8d3770-835a-4b7b-b977-65d8042f20fa.assettoscatemplate" CURR_ARTIFACT_UUID = "fe8d3770-835a-4b7b-b977-65d8042f20fa.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extvl-template.yml,0,fe8d3770-835a-4b7b-b977-65d8042f20fa.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bde5d5c3-2c56-4912-b705-6eee1d9c838d TargetServiceName=null EntryTimestamp=2021-08-11T16:09:42.531Z InvocationID=6fac35d8-893a-4ca0-984f-abd7d3537cbb ErrorCode=500 Starting to create Csar for component ExtVL 2021-08-11T16:09:46.267Z [qtp2045766957-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ExtVL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "abfc0f07-aea8-4edb-9cb6-68a85a9d3ca8" INVARIANT_UUID = "a72ce58a-d31d-4aa7-88f8-8c9c09ab9101" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "fe8d3770-835a-4b7b-b977-65d8042f20fa.assettoscatemplate" CURR_ARTIFACT_UUID = "fe8d3770-835a-4b7b-b977-65d8042f20fa.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extvl-template.yml,0,fe8d3770-835a-4b7b-b977-65d8042f20fa.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bde5d5c3-2c56-4912-b705-6eee1d9c838d TargetServiceName=null EntryTimestamp=2021-08-11T16:09:42.531Z InvocationID=6fac35d8-893a-4ca0-984f-abd7d3537cbb ErrorCode=500 Ended create Csar for component ExtVL 2021-08-11T16:09:46.267Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ExtVL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "abfc0f07-aea8-4edb-9cb6-68a85a9d3ca8" INVARIANT_UUID = "a72ce58a-d31d-4aa7-88f8-8c9c09ab9101" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "fe8d3770-835a-4b7b-b977-65d8042f20fa.assettoscatemplate" CURR_ARTIFACT_UUID = "fe8d3770-835a-4b7b-b977-65d8042f20fa.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extvl-template.yml,0,fe8d3770-835a-4b7b-b977-65d8042f20fa.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bde5d5c3-2c56-4912-b705-6eee1d9c838d TargetServiceName=null EntryTimestamp=2021-08-11T16:09:42.531Z InvocationID=6fac35d8-893a-4ca0-984f-abd7d3537cbb ErrorCode=500 lock resource with id fe8d3770-835a-4b7b-b977-65d8042f20fa 2021-08-11T16:09:46.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=982094ac-db4c-4535-bd7b-b0d443684988 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:09:46.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=982094ac-db4c-4535-bd7b-b0d443684988 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6692d3a 2021-08-11T16:09:46.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=982094ac-db4c-4535-bd7b-b0d443684988 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:09:46.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=982094ac-db4c-4535-bd7b-b0d443684988 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:09:47.892Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ExtVL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "abfc0f07-aea8-4edb-9cb6-68a85a9d3ca8" INVARIANT_UUID = "a72ce58a-d31d-4aa7-88f8-8c9c09ab9101" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "fe8d3770-835a-4b7b-b977-65d8042f20fa.assettoscacsar" CURR_ARTIFACT_UUID = "fe8d3770-835a-4b7b-b977-65d8042f20fa.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Extvl-csar.csar,0,fe8d3770-835a-4b7b-b977-65d8042f20fa.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5361 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=50e154f1-fa4e-4841-bc34-abbe465e93dc TargetServiceName=null EntryTimestamp=2021-08-11T16:09:42.531Z InvocationID=6fac35d8-893a-4ca0-984f-abd7d3537cbb ErrorCode=500 LogTimestamp=2021-08-11T16:09:47.892185Z ResponseDesc=Created Exiting. 2021-08-11T16:09:47.893Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:09:47.949Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=48b2815e-8a0f-4e24-90b7-36a44e9b3126 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:09:47.949283Z InvocationID=00e16c7b-76cf-425f-bf93-f8262148b774 LogTimestamp=2021-08-11T16:09:47.949350Z Entering 2021-08-11T16:09:47.949Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=48b2815e-8a0f-4e24-90b7-36a44e9b3126 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:09:47.949283Z InvocationID=00e16c7b-76cf-425f-bf93-f8262148b774 LogTimestamp=2021-08-11T16:09:47.949350Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:09:48.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:09:48.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:09:48.558Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:47.950Z InvocationID=b4af76a1-1931-41c9-b72e-abce0e25a7f6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:48.558Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:47.950Z InvocationID=b4af76a1-1931-41c9-b72e-abce0e25a7f6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:48.558Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:47.950Z InvocationID=b4af76a1-1931-41c9-b72e-abce0e25a7f6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:48.558Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:47.950Z InvocationID=b4af76a1-1931-41c9-b72e-abce0e25a7f6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:48.558Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:47.950Z InvocationID=b4af76a1-1931-41c9-b72e-abce0e25a7f6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Internalvl 2021-08-11T16:09:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:09:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:09:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:09:50.053Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bbfbbcdb-fb1e-457b-8762-2ae82abff47c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:09:50.053Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bbfbbcdb-fb1e-457b-8762-2ae82abff47c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@14615847 2021-08-11T16:09:50.053Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bbfbbcdb-fb1e-457b-8762-2ae82abff47c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:09:50.053Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bbfbbcdb-fb1e-457b-8762-2ae82abff47c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:09:50.547Z [qtp2045766957-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "InternalVL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "e68acf0b-5f1b-415f-acb0-45b934aab70b" INVARIANT_UUID = "44ebbec1-27ba-4ac6-9db3-e7fe55723fc9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.internalVL" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e68acf0b-5f1b-415f-acb0-45b934aab70b EntryTimestamp=2021-08-11T16:09:47.950Z InvocationID=b4af76a1-1931-41c9-b72e-abce0e25a7f6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:09:50.548Z [qtp2045766957-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "InternalVL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "e68acf0b-5f1b-415f-acb0-45b934aab70b" INVARIANT_UUID = "44ebbec1-27ba-4ac6-9db3-e7fe55723fc9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.internalVL" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e68acf0b-5f1b-415f-acb0-45b934aab70b TargetServiceName=null EntryTimestamp=2021-08-11T16:09:47.950Z InvocationID=b4af76a1-1931-41c9-b72e-abce0e25a7f6 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:09:51.276Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "InternalVL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "e68acf0b-5f1b-415f-acb0-45b934aab70b" INVARIANT_UUID = "44ebbec1-27ba-4ac6-9db3-e7fe55723fc9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.internalVL" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e68acf0b-5f1b-415f-acb0-45b934aab70b TargetServiceName=null EntryTimestamp=2021-08-11T16:09:47.950Z InvocationID=b4af76a1-1931-41c9-b72e-abce0e25a7f6 ErrorCode=500 lock resource with id 45329156-ea18-46d6-8e73-97cf310b80d3 2021-08-11T16:09:53.091Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e275a72-0fe6-465c-9aa8-f7e048435574 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:09:53.091Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e275a72-0fe6-465c-9aa8-f7e048435574 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7dac2ab5 2021-08-11T16:09:53.091Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e275a72-0fe6-465c-9aa8-f7e048435574 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:09:53.091Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e275a72-0fe6-465c-9aa8-f7e048435574 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:09:53.447Z [qtp2045766957-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "InternalVL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "e68acf0b-5f1b-415f-acb0-45b934aab70b" INVARIANT_UUID = "44ebbec1-27ba-4ac6-9db3-e7fe55723fc9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "45329156-ea18-46d6-8e73-97cf310b80d3.assettoscatemplate" CURR_ARTIFACT_UUID = "45329156-ea18-46d6-8e73-97cf310b80d3.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Internalvl-template.yml,0,45329156-ea18-46d6-8e73-97cf310b80d3.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3bebe5a0-1238-4716-9513-a409e21699e7 TargetServiceName=null EntryTimestamp=2021-08-11T16:09:47.950Z InvocationID=b4af76a1-1931-41c9-b72e-abce0e25a7f6 ErrorCode=500 Starting to create Csar for component InternalVL 2021-08-11T16:09:53.812Z [qtp2045766957-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "InternalVL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "e68acf0b-5f1b-415f-acb0-45b934aab70b" INVARIANT_UUID = "44ebbec1-27ba-4ac6-9db3-e7fe55723fc9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "45329156-ea18-46d6-8e73-97cf310b80d3.assettoscatemplate" CURR_ARTIFACT_UUID = "45329156-ea18-46d6-8e73-97cf310b80d3.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Internalvl-template.yml,0,45329156-ea18-46d6-8e73-97cf310b80d3.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3bebe5a0-1238-4716-9513-a409e21699e7 TargetServiceName=null EntryTimestamp=2021-08-11T16:09:47.950Z InvocationID=b4af76a1-1931-41c9-b72e-abce0e25a7f6 ErrorCode=500 Ended create Csar for component InternalVL 2021-08-11T16:09:53.812Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "InternalVL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "e68acf0b-5f1b-415f-acb0-45b934aab70b" INVARIANT_UUID = "44ebbec1-27ba-4ac6-9db3-e7fe55723fc9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "45329156-ea18-46d6-8e73-97cf310b80d3.assettoscatemplate" CURR_ARTIFACT_UUID = "45329156-ea18-46d6-8e73-97cf310b80d3.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Internalvl-template.yml,0,45329156-ea18-46d6-8e73-97cf310b80d3.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3bebe5a0-1238-4716-9513-a409e21699e7 TargetServiceName=null EntryTimestamp=2021-08-11T16:09:47.950Z InvocationID=b4af76a1-1931-41c9-b72e-abce0e25a7f6 ErrorCode=500 lock resource with id 45329156-ea18-46d6-8e73-97cf310b80d3 2021-08-11T16:09:55.787Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "InternalVL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "e68acf0b-5f1b-415f-acb0-45b934aab70b" INVARIANT_UUID = "44ebbec1-27ba-4ac6-9db3-e7fe55723fc9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "45329156-ea18-46d6-8e73-97cf310b80d3.assettoscacsar" CURR_ARTIFACT_UUID = "45329156-ea18-46d6-8e73-97cf310b80d3.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Internalvl-csar.csar,0,45329156-ea18-46d6-8e73-97cf310b80d3.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=7836 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=348dd92a-4d95-46df-9e5f-ea3432d14071 TargetServiceName=null EntryTimestamp=2021-08-11T16:09:47.950Z InvocationID=b4af76a1-1931-41c9-b72e-abce0e25a7f6 ErrorCode=500 LogTimestamp=2021-08-11T16:09:55.786916Z ResponseDesc=Created Exiting. 2021-08-11T16:09:55.788Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:09:55.867Z [qtp2045766957-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=deeb881e-f8bf-44ad-aa49-cb30f388eb94 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:09:55.866618Z InvocationID=ede9a471-678b-49b1-b63b-1b157a2acd93 LogTimestamp=2021-08-11T16:09:55.866714Z Entering 2021-08-11T16:09:55.867Z [qtp2045766957-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=deeb881e-f8bf-44ad-aa49-cb30f388eb94 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:09:55.866618Z InvocationID=ede9a471-678b-49b1-b63b-1b157a2acd93 LogTimestamp=2021-08-11T16:09:55.866714Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:09:56.564Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bdd8b57d-47ae-4d12-9c4f-4b0fffdf6089 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:09:56.564Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bdd8b57d-47ae-4d12-9c4f-4b0fffdf6089 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@687b70e3 2021-08-11T16:09:56.564Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bdd8b57d-47ae-4d12-9c4f-4b0fffdf6089 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:09:56.564Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bdd8b57d-47ae-4d12-9c4f-4b0fffdf6089 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:09:57.566Z [qtp2045766957-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:55.867Z InvocationID=fd277256-1778-416c-95d3-d6deb86fa76e PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:57.567Z [qtp2045766957-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:55.867Z InvocationID=fd277256-1778-416c-95d3-d6deb86fa76e PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:57.567Z [qtp2045766957-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:55.867Z InvocationID=fd277256-1778-416c-95d3-d6deb86fa76e PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:57.567Z [qtp2045766957-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:55.867Z InvocationID=fd277256-1778-416c-95d3-d6deb86fa76e PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:09:57.685Z [qtp2045766957-20] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:09:55.867Z InvocationID=fd277256-1778-416c-95d3-d6deb86fa76e PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Extcp 2021-08-11T16:09:58.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3a0ee269-ddd6-4512-a57a-14cede790244 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:09:58.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3a0ee269-ddd6-4512-a57a-14cede790244 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@27b4902b 2021-08-11T16:09:58.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3a0ee269-ddd6-4512-a57a-14cede790244 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:09:58.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3a0ee269-ddd6-4512-a57a-14cede790244 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:09:58.970Z [qtp2045766957-20] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ExtCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "3c4fb6cd-4442-4da5-a5a1-29100bfc1aff" INVARIANT_UUID = "06ca4419-4418-494b-a380-08daa7955f3a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.extCP" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3c4fb6cd-4442-4da5-a5a1-29100bfc1aff EntryTimestamp=2021-08-11T16:09:55.867Z InvocationID=fd277256-1778-416c-95d3-d6deb86fa76e PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:09:58.971Z [qtp2045766957-20] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ExtCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "3c4fb6cd-4442-4da5-a5a1-29100bfc1aff" INVARIANT_UUID = "06ca4419-4418-494b-a380-08daa7955f3a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.extCP" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3c4fb6cd-4442-4da5-a5a1-29100bfc1aff TargetServiceName=null EntryTimestamp=2021-08-11T16:09:55.867Z InvocationID=fd277256-1778-416c-95d3-d6deb86fa76e ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:09:59.310Z [qtp2045766957-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ExtCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "3c4fb6cd-4442-4da5-a5a1-29100bfc1aff" INVARIANT_UUID = "06ca4419-4418-494b-a380-08daa7955f3a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.extCP" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3c4fb6cd-4442-4da5-a5a1-29100bfc1aff TargetServiceName=null EntryTimestamp=2021-08-11T16:09:55.867Z InvocationID=fd277256-1778-416c-95d3-d6deb86fa76e ErrorCode=500 lock resource with id 7d9e3765-38d6-4d89-904a-88bf0db6857f 2021-08-11T16:10:00.923Z [qtp2045766957-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ExtCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "3c4fb6cd-4442-4da5-a5a1-29100bfc1aff" INVARIANT_UUID = "06ca4419-4418-494b-a380-08daa7955f3a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "7d9e3765-38d6-4d89-904a-88bf0db6857f.assettoscatemplate" CURR_ARTIFACT_UUID = "7d9e3765-38d6-4d89-904a-88bf0db6857f.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extcp-template.yml,0,7d9e3765-38d6-4d89-904a-88bf0db6857f.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d42e2a16-db01-4537-b815-0ce806752840 TargetServiceName=null EntryTimestamp=2021-08-11T16:09:55.867Z InvocationID=fd277256-1778-416c-95d3-d6deb86fa76e ErrorCode=500 Starting to create Csar for component ExtCP 2021-08-11T16:10:01.126Z [qtp2045766957-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ExtCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "3c4fb6cd-4442-4da5-a5a1-29100bfc1aff" INVARIANT_UUID = "06ca4419-4418-494b-a380-08daa7955f3a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "7d9e3765-38d6-4d89-904a-88bf0db6857f.assettoscatemplate" CURR_ARTIFACT_UUID = "7d9e3765-38d6-4d89-904a-88bf0db6857f.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extcp-template.yml,0,7d9e3765-38d6-4d89-904a-88bf0db6857f.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d42e2a16-db01-4537-b815-0ce806752840 TargetServiceName=null EntryTimestamp=2021-08-11T16:09:55.867Z InvocationID=fd277256-1778-416c-95d3-d6deb86fa76e ErrorCode=500 Ended create Csar for component ExtCP 2021-08-11T16:10:01.127Z [qtp2045766957-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ExtCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "3c4fb6cd-4442-4da5-a5a1-29100bfc1aff" INVARIANT_UUID = "06ca4419-4418-494b-a380-08daa7955f3a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "7d9e3765-38d6-4d89-904a-88bf0db6857f.assettoscatemplate" CURR_ARTIFACT_UUID = "7d9e3765-38d6-4d89-904a-88bf0db6857f.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extcp-template.yml,0,7d9e3765-38d6-4d89-904a-88bf0db6857f.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d42e2a16-db01-4537-b815-0ce806752840 TargetServiceName=null EntryTimestamp=2021-08-11T16:09:55.867Z InvocationID=fd277256-1778-416c-95d3-d6deb86fa76e ErrorCode=500 lock resource with id 7d9e3765-38d6-4d89-904a-88bf0db6857f 2021-08-11T16:10:01.883Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=26b5e108-df66-4223-8b5b-ebe4d862f065 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:10:01.883Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=26b5e108-df66-4223-8b5b-ebe4d862f065 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@390957fa 2021-08-11T16:10:01.883Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=26b5e108-df66-4223-8b5b-ebe4d862f065 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:10:01.883Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=26b5e108-df66-4223-8b5b-ebe4d862f065 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:10:02.855Z [qtp2045766957-20] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ExtCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "3c4fb6cd-4442-4da5-a5a1-29100bfc1aff" INVARIANT_UUID = "06ca4419-4418-494b-a380-08daa7955f3a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "7d9e3765-38d6-4d89-904a-88bf0db6857f.assettoscacsar" CURR_ARTIFACT_UUID = "7d9e3765-38d6-4d89-904a-88bf0db6857f.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Extcp-csar.csar,0,7d9e3765-38d6-4d89-904a-88bf0db6857f.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=6987 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ad4c9b17-2e11-4bc6-b395-9ebaee3301fc TargetServiceName=null EntryTimestamp=2021-08-11T16:09:55.867Z InvocationID=fd277256-1778-416c-95d3-d6deb86fa76e ErrorCode=500 LogTimestamp=2021-08-11T16:10:02.854432Z ResponseDesc=Created Exiting. 2021-08-11T16:10:02.855Z [qtp2045766957-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:10:02.880Z [qtp2045766957-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=77f185f1-28c5-481e-827d-6426f59d7718 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:10:02.879759Z InvocationID=67473581-be16-42fd-b793-ed5b28c11338 LogTimestamp=2021-08-11T16:10:02.879838Z Entering 2021-08-11T16:10:02.880Z [qtp2045766957-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=77f185f1-28c5-481e-827d-6426f59d7718 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:10:02.879759Z InvocationID=67473581-be16-42fd-b793-ed5b28c11338 LogTimestamp=2021-08-11T16:10:02.879838Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:10:03.169Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:02.880Z InvocationID=125eb0d8-d257-421e-ad49-73443724efcf PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:03.169Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:02.880Z InvocationID=125eb0d8-d257-421e-ad49-73443724efcf PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:03.169Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:02.880Z InvocationID=125eb0d8-d257-421e-ad49-73443724efcf PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:03.169Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:02.880Z InvocationID=125eb0d8-d257-421e-ad49-73443724efcf PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:03.219Z [qtp2045766957-11] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:02.880Z InvocationID=125eb0d8-d257-421e-ad49-73443724efcf PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Vl 2021-08-11T16:10:03.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:10:03.316Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:10:04.257Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:10:04.589Z [qtp2045766957-11] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "ff2cc9c9-5f9a-42bc-a899-7fb4e9e30d42" INVARIANT_UUID = "2f5297af-ed7e-4bf8-ac31-6934bf2b0597" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.VL" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ff2cc9c9-5f9a-42bc-a899-7fb4e9e30d42 EntryTimestamp=2021-08-11T16:10:02.880Z InvocationID=125eb0d8-d257-421e-ad49-73443724efcf PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:10:04.590Z [qtp2045766957-11] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "ff2cc9c9-5f9a-42bc-a899-7fb4e9e30d42" INVARIANT_UUID = "2f5297af-ed7e-4bf8-ac31-6934bf2b0597" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.VL" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ff2cc9c9-5f9a-42bc-a899-7fb4e9e30d42 TargetServiceName=null EntryTimestamp=2021-08-11T16:10:02.880Z InvocationID=125eb0d8-d257-421e-ad49-73443724efcf ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:10:05.063Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c48ec4bf-481d-4cca-b693-dbf1719b540c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:10:05.063Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c48ec4bf-481d-4cca-b693-dbf1719b540c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@25ec2bec 2021-08-11T16:10:05.063Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c48ec4bf-481d-4cca-b693-dbf1719b540c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:10:05.063Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c48ec4bf-481d-4cca-b693-dbf1719b540c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:10:05.291Z [qtp2045766957-11] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "ff2cc9c9-5f9a-42bc-a899-7fb4e9e30d42" INVARIANT_UUID = "2f5297af-ed7e-4bf8-ac31-6934bf2b0597" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.VL" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ff2cc9c9-5f9a-42bc-a899-7fb4e9e30d42 TargetServiceName=null EntryTimestamp=2021-08-11T16:10:02.880Z InvocationID=125eb0d8-d257-421e-ad49-73443724efcf ErrorCode=500 lock resource with id a51bb4bf-9ec9-4233-8768-5db035252b4b 2021-08-11T16:10:07.080Z [qtp2045766957-11] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "ff2cc9c9-5f9a-42bc-a899-7fb4e9e30d42" INVARIANT_UUID = "2f5297af-ed7e-4bf8-ac31-6934bf2b0597" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a51bb4bf-9ec9-4233-8768-5db035252b4b.assettoscatemplate" CURR_ARTIFACT_UUID = "a51bb4bf-9ec9-4233-8768-5db035252b4b.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vl-template.yml,0,a51bb4bf-9ec9-4233-8768-5db035252b4b.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=62bf946e-46ec-484f-a47c-4a665b191501 TargetServiceName=null EntryTimestamp=2021-08-11T16:10:02.880Z InvocationID=125eb0d8-d257-421e-ad49-73443724efcf ErrorCode=500 Starting to create Csar for component VL 2021-08-11T16:10:07.262Z [qtp2045766957-11] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "ff2cc9c9-5f9a-42bc-a899-7fb4e9e30d42" INVARIANT_UUID = "2f5297af-ed7e-4bf8-ac31-6934bf2b0597" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a51bb4bf-9ec9-4233-8768-5db035252b4b.assettoscatemplate" CURR_ARTIFACT_UUID = "a51bb4bf-9ec9-4233-8768-5db035252b4b.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vl-template.yml,0,a51bb4bf-9ec9-4233-8768-5db035252b4b.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=62bf946e-46ec-484f-a47c-4a665b191501 TargetServiceName=null EntryTimestamp=2021-08-11T16:10:02.880Z InvocationID=125eb0d8-d257-421e-ad49-73443724efcf ErrorCode=500 Ended create Csar for component VL 2021-08-11T16:10:07.262Z [qtp2045766957-11] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "ff2cc9c9-5f9a-42bc-a899-7fb4e9e30d42" INVARIANT_UUID = "2f5297af-ed7e-4bf8-ac31-6934bf2b0597" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a51bb4bf-9ec9-4233-8768-5db035252b4b.assettoscatemplate" CURR_ARTIFACT_UUID = "a51bb4bf-9ec9-4233-8768-5db035252b4b.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vl-template.yml,0,a51bb4bf-9ec9-4233-8768-5db035252b4b.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=62bf946e-46ec-484f-a47c-4a665b191501 TargetServiceName=null EntryTimestamp=2021-08-11T16:10:02.880Z InvocationID=125eb0d8-d257-421e-ad49-73443724efcf ErrorCode=500 lock resource with id a51bb4bf-9ec9-4233-8768-5db035252b4b 2021-08-11T16:10:07.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e3d3988-09b1-40d1-8393-1e9495879664 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:10:07.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e3d3988-09b1-40d1-8393-1e9495879664 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1623bbda 2021-08-11T16:10:07.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e3d3988-09b1-40d1-8393-1e9495879664 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:10:07.972Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e3d3988-09b1-40d1-8393-1e9495879664 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:10:08.970Z [qtp2045766957-11] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "ff2cc9c9-5f9a-42bc-a899-7fb4e9e30d42" INVARIANT_UUID = "2f5297af-ed7e-4bf8-ac31-6934bf2b0597" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a51bb4bf-9ec9-4233-8768-5db035252b4b.assettoscacsar" CURR_ARTIFACT_UUID = "a51bb4bf-9ec9-4233-8768-5db035252b4b.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Vl-csar.csar,0,a51bb4bf-9ec9-4233-8768-5db035252b4b.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=6089 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c92dc17f-6647-44a6-9a59-0128f662826a TargetServiceName=null EntryTimestamp=2021-08-11T16:10:02.880Z InvocationID=125eb0d8-d257-421e-ad49-73443724efcf ErrorCode=500 LogTimestamp=2021-08-11T16:10:08.969625Z ResponseDesc=Created Exiting. 2021-08-11T16:10:08.971Z [qtp2045766957-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:10:08.999Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=549f64d8-3fc6-499d-ab42-80a8b2f3d75f InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:10:08.999191Z InvocationID=49345498-4df9-45f3-8e1e-9c3f4890e47f LogTimestamp=2021-08-11T16:10:08.999321Z Entering 2021-08-11T16:10:08.999Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=549f64d8-3fc6-499d-ab42-80a8b2f3d75f InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:10:08.999191Z InvocationID=49345498-4df9-45f3-8e1e-9c3f4890e47f LogTimestamp=2021-08-11T16:10:08.999321Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:10:09.291Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:09.000Z InvocationID=caf0c2d0-e512-4826-ad22-be84f2db9e0b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:09.291Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:09.000Z InvocationID=caf0c2d0-e512-4826-ad22-be84f2db9e0b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:09.291Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:09.000Z InvocationID=caf0c2d0-e512-4826-ad22-be84f2db9e0b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:09.291Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:09.000Z InvocationID=caf0c2d0-e512-4826-ad22-be84f2db9e0b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:09.305Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:09.000Z InvocationID=caf0c2d0-e512-4826-ad22-be84f2db9e0b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name VlEline 2021-08-11T16:10:10.862Z [qtp2045766957-14] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VL ELINE" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "443b01db-9259-4396-a9d2-78435b51ef4e" INVARIANT_UUID = "16fcd01c-b801-4d14-a68a-fb2cae6df8ec" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.ELine" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=443b01db-9259-4396-a9d2-78435b51ef4e EntryTimestamp=2021-08-11T16:10:09.000Z InvocationID=caf0c2d0-e512-4826-ad22-be84f2db9e0b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:10:10.863Z [qtp2045766957-14] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VL ELINE" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "443b01db-9259-4396-a9d2-78435b51ef4e" INVARIANT_UUID = "16fcd01c-b801-4d14-a68a-fb2cae6df8ec" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.ELine" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=443b01db-9259-4396-a9d2-78435b51ef4e TargetServiceName=null EntryTimestamp=2021-08-11T16:10:09.000Z InvocationID=caf0c2d0-e512-4826-ad22-be84f2db9e0b ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:10:10.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3ae37619-9a07-43fa-a4fe-126e07f41274 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:10:10.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3ae37619-9a07-43fa-a4fe-126e07f41274 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@159544fd 2021-08-11T16:10:10.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3ae37619-9a07-43fa-a4fe-126e07f41274 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:10:10.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3ae37619-9a07-43fa-a4fe-126e07f41274 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:10:11.252Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VL ELINE" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "443b01db-9259-4396-a9d2-78435b51ef4e" INVARIANT_UUID = "16fcd01c-b801-4d14-a68a-fb2cae6df8ec" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.ELine" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=443b01db-9259-4396-a9d2-78435b51ef4e TargetServiceName=null EntryTimestamp=2021-08-11T16:10:09.000Z InvocationID=caf0c2d0-e512-4826-ad22-be84f2db9e0b ErrorCode=500 lock resource with id 9fb84591-c826-4fff-aa9e-cae581a337f5 2021-08-11T16:10:12.867Z [qtp2045766957-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VL ELINE" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "443b01db-9259-4396-a9d2-78435b51ef4e" INVARIANT_UUID = "16fcd01c-b801-4d14-a68a-fb2cae6df8ec" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "9fb84591-c826-4fff-aa9e-cae581a337f5.assettoscatemplate" CURR_ARTIFACT_UUID = "9fb84591-c826-4fff-aa9e-cae581a337f5.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VlEline-template.yml,0,9fb84591-c826-4fff-aa9e-cae581a337f5.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1db2aa3e-a3d2-4923-80bc-804327deb353 TargetServiceName=null EntryTimestamp=2021-08-11T16:10:09.000Z InvocationID=caf0c2d0-e512-4826-ad22-be84f2db9e0b ErrorCode=500 Starting to create Csar for component VL ELINE 2021-08-11T16:10:13.097Z [qtp2045766957-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VL ELINE" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "443b01db-9259-4396-a9d2-78435b51ef4e" INVARIANT_UUID = "16fcd01c-b801-4d14-a68a-fb2cae6df8ec" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "9fb84591-c826-4fff-aa9e-cae581a337f5.assettoscatemplate" CURR_ARTIFACT_UUID = "9fb84591-c826-4fff-aa9e-cae581a337f5.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VlEline-template.yml,0,9fb84591-c826-4fff-aa9e-cae581a337f5.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1db2aa3e-a3d2-4923-80bc-804327deb353 TargetServiceName=null EntryTimestamp=2021-08-11T16:10:09.000Z InvocationID=caf0c2d0-e512-4826-ad22-be84f2db9e0b ErrorCode=500 Ended create Csar for component VL ELINE 2021-08-11T16:10:13.098Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VL ELINE" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "443b01db-9259-4396-a9d2-78435b51ef4e" INVARIANT_UUID = "16fcd01c-b801-4d14-a68a-fb2cae6df8ec" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "9fb84591-c826-4fff-aa9e-cae581a337f5.assettoscatemplate" CURR_ARTIFACT_UUID = "9fb84591-c826-4fff-aa9e-cae581a337f5.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VlEline-template.yml,0,9fb84591-c826-4fff-aa9e-cae581a337f5.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1db2aa3e-a3d2-4923-80bc-804327deb353 TargetServiceName=null EntryTimestamp=2021-08-11T16:10:09.000Z InvocationID=caf0c2d0-e512-4826-ad22-be84f2db9e0b ErrorCode=500 lock resource with id 9fb84591-c826-4fff-aa9e-cae581a337f5 2021-08-11T16:10:13.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b8f9c2c-c498-4239-84f1-05d2d13e4df8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:10:13.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b8f9c2c-c498-4239-84f1-05d2d13e4df8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@66506732 2021-08-11T16:10:13.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b8f9c2c-c498-4239-84f1-05d2d13e4df8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:10:13.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b8f9c2c-c498-4239-84f1-05d2d13e4df8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:10:14.751Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VL ELINE" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "443b01db-9259-4396-a9d2-78435b51ef4e" INVARIANT_UUID = "16fcd01c-b801-4d14-a68a-fb2cae6df8ec" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "9fb84591-c826-4fff-aa9e-cae581a337f5.assettoscacsar" CURR_ARTIFACT_UUID = "9fb84591-c826-4fff-aa9e-cae581a337f5.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VlEline-csar.csar,0,9fb84591-c826-4fff-aa9e-cae581a337f5.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5750 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ab8ddd6a-735a-47bf-8325-bdfcfef5447f TargetServiceName=null EntryTimestamp=2021-08-11T16:10:09.000Z InvocationID=caf0c2d0-e512-4826-ad22-be84f2db9e0b ErrorCode=500 LogTimestamp=2021-08-11T16:10:14.750284Z ResponseDesc=Created Exiting. 2021-08-11T16:10:14.751Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:10:14.777Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=8f54f06f-0288-4c22-bbcc-e14791c782d8 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:10:14.776445Z InvocationID=90fce4f7-f628-456d-b600-fffeb753a8e8 LogTimestamp=2021-08-11T16:10:14.776547Z Entering 2021-08-11T16:10:14.777Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=8f54f06f-0288-4c22-bbcc-e14791c782d8 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:10:14.776445Z InvocationID=90fce4f7-f628-456d-b600-fffeb753a8e8 LogTimestamp=2021-08-11T16:10:14.776547Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:10:15.105Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:14.777Z InvocationID=288b78f2-6911-4b0f-8feb-dc44da9c13a5 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:15.105Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:14.777Z InvocationID=288b78f2-6911-4b0f-8feb-dc44da9c13a5 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:15.105Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:14.777Z InvocationID=288b78f2-6911-4b0f-8feb-dc44da9c13a5 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:15.105Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:14.777Z InvocationID=288b78f2-6911-4b0f-8feb-dc44da9c13a5 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:15.105Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:14.777Z InvocationID=288b78f2-6911-4b0f-8feb-dc44da9c13a5 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Abstractsubstitute 2021-08-11T16:10:16.603Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "AbstractSubstitute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "cbc98b0c-c873-49af-bf37-20dfb743ae3e" INVARIANT_UUID = "74494457-c6e8-4db3-92a5-6ac79774719b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.abstract.nodes.AbstractSubstitute" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=cbc98b0c-c873-49af-bf37-20dfb743ae3e EntryTimestamp=2021-08-11T16:10:14.777Z InvocationID=288b78f2-6911-4b0f-8feb-dc44da9c13a5 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id 1392bb4b-5340-4b44-8e97-fe4fce40f034 2021-08-11T16:10:16.700Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6cd8b99e-1256-41a3-9d63-a1c36ab9d444 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:10:16.700Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6cd8b99e-1256-41a3-9d63-a1c36ab9d444 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@22a7835f 2021-08-11T16:10:16.700Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6cd8b99e-1256-41a3-9d63-a1c36ab9d444 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:10:16.700Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6cd8b99e-1256-41a3-9d63-a1c36ab9d444 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:10:18.277Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "AbstractSubstitute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "cbc98b0c-c873-49af-bf37-20dfb743ae3e" INVARIANT_UUID = "74494457-c6e8-4db3-92a5-6ac79774719b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "1392bb4b-5340-4b44-8e97-fe4fce40f034.assettoscatemplate" CURR_ARTIFACT_UUID = "1392bb4b-5340-4b44-8e97-fe4fce40f034.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Abstractsubstitute-template.yml,0,1392bb4b-5340-4b44-8e97-fe4fce40f034.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=3499 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=0a24cde4-4807-4b92-be8a-52350900930d EntryTimestamp=2021-08-11T16:10:14.777Z InvocationID=288b78f2-6911-4b0f-8feb-dc44da9c13a5 LogTimestamp=2021-08-11T16:10:18.276566Z ResponseDesc=Created Exiting. 2021-08-11T16:10:18.278Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:10:18.298Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=9d126fa9-b13a-4cdc-93be-360c115f2466 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:10:18.297629Z InvocationID=3617ae16-53ee-4100-9a14-fb7db6397b0b LogTimestamp=2021-08-11T16:10:18.297707Z Entering 2021-08-11T16:10:18.298Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=9d126fa9-b13a-4cdc-93be-360c115f2466 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:10:18.297629Z InvocationID=3617ae16-53ee-4100-9a14-fb7db6397b0b LogTimestamp=2021-08-11T16:10:18.297707Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:10:18.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:10:18.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:10:18.619Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:18.298Z InvocationID=132a9f90-c1f9-4962-aa7d-49fc9ee4562f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:18.619Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:18.298Z InvocationID=132a9f90-c1f9-4962-aa7d-49fc9ee4562f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:18.619Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:18.298Z InvocationID=132a9f90-c1f9-4962-aa7d-49fc9ee4562f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:18.619Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:18.298Z InvocationID=132a9f90-c1f9-4962-aa7d-49fc9ee4562f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:18.620Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:18.298Z InvocationID=132a9f90-c1f9-4962-aa7d-49fc9ee4562f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name GenericVfc 2021-08-11T16:10:19.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=48fafdbc-726e-4d17-9da5-84984c52a7d8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:10:19.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=48fafdbc-726e-4d17-9da5-84984c52a7d8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@295101ba 2021-08-11T16:10:19.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=48fafdbc-726e-4d17-9da5-84984c52a7d8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:10:19.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=48fafdbc-726e-4d17-9da5-84984c52a7d8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:10:20.176Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Generic_VFC" RESOURCE_TYPE = "ABSTRACT" SERVICE_INSTANCE_ID = "d8c86d1a-6845-40a3-8972-d075a7828489" INVARIANT_UUID = "f950c054-2553-41e3-b8d3-cd161b9812fc" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.abstract.nodes.VFC" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d8c86d1a-6845-40a3-8972-d075a7828489 EntryTimestamp=2021-08-11T16:10:18.298Z InvocationID=132a9f90-c1f9-4962-aa7d-49fc9ee4562f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id 4a49959c-46ef-46ee-9ee5-44d480ed5406 2021-08-11T16:10:21.801Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Generic_VFC" RESOURCE_TYPE = "ABSTRACT" SERVICE_INSTANCE_ID = "d8c86d1a-6845-40a3-8972-d075a7828489" INVARIANT_UUID = "f950c054-2553-41e3-b8d3-cd161b9812fc" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "4a49959c-46ef-46ee-9ee5-44d480ed5406.assettoscatemplate" CURR_ARTIFACT_UUID = "4a49959c-46ef-46ee-9ee5-44d480ed5406.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-GenericVfc-template.yml,0,4a49959c-46ef-46ee-9ee5-44d480ed5406.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=3503 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b8c0834a-6674-4076-b18b-801a79ec033f EntryTimestamp=2021-08-11T16:10:18.298Z InvocationID=132a9f90-c1f9-4962-aa7d-49fc9ee4562f LogTimestamp=2021-08-11T16:10:21.801268Z ResponseDesc=Created Exiting. 2021-08-11T16:10:21.803Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:10:21.820Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=0f46d156-59ee-403a-99ad-1041eb0769ef InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:10:21.819641Z InvocationID=1126d3d5-8f3b-4ae4-8f54-2ea1d4ce22df LogTimestamp=2021-08-11T16:10:21.819741Z Entering 2021-08-11T16:10:21.820Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=0f46d156-59ee-403a-99ad-1041eb0769ef InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:10:21.819641Z InvocationID=1126d3d5-8f3b-4ae4-8f54-2ea1d4ce22df LogTimestamp=2021-08-11T16:10:21.819741Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:10:22.159Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:21.821Z InvocationID=197f3acf-3660-492c-9508-8855d792fecc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:22.159Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:21.821Z InvocationID=197f3acf-3660-492c-9508-8855d792fecc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:22.159Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:21.821Z InvocationID=197f3acf-3660-492c-9508-8855d792fecc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:22.159Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:21.821Z InvocationID=197f3acf-3660-492c-9508-8855d792fecc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:22.159Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:21.821Z InvocationID=197f3acf-3660-492c-9508-8855d792fecc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name GenericVf 2021-08-11T16:10:22.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=928b046c-8314-4f52-83c2-d25c94590453 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:10:22.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=928b046c-8314-4f52-83c2-d25c94590453 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@16dc0ac1 2021-08-11T16:10:22.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=928b046c-8314-4f52-83c2-d25c94590453 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:10:22.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=928b046c-8314-4f52-83c2-d25c94590453 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:10:23.989Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Generic_VF" RESOURCE_TYPE = "ABSTRACT" SERVICE_INSTANCE_ID = "fea1dec9-9a6e-43af-9749-a550fb5a12a2" INVARIANT_UUID = "08cb788a-c355-4eef-964c-781b7b757cd6" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.abstract.nodes.VF" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=fea1dec9-9a6e-43af-9749-a550fb5a12a2 EntryTimestamp=2021-08-11T16:10:21.821Z InvocationID=197f3acf-3660-492c-9508-8855d792fecc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id afce8c86-0af8-42d5-a8f9-43745ef8e184 2021-08-11T16:10:25.674Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Generic_VF" RESOURCE_TYPE = "ABSTRACT" SERVICE_INSTANCE_ID = "fea1dec9-9a6e-43af-9749-a550fb5a12a2" INVARIANT_UUID = "08cb788a-c355-4eef-964c-781b7b757cd6" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "afce8c86-0af8-42d5-a8f9-43745ef8e184.assettoscatemplate" CURR_ARTIFACT_UUID = "afce8c86-0af8-42d5-a8f9-43745ef8e184.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-GenericVf-template.yml,0,afce8c86-0af8-42d5-a8f9-43745ef8e184.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=3853 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=80ce49f3-e444-453f-a7a9-18c9cd873bfa EntryTimestamp=2021-08-11T16:10:21.821Z InvocationID=197f3acf-3660-492c-9508-8855d792fecc LogTimestamp=2021-08-11T16:10:25.674217Z ResponseDesc=Created Exiting. 2021-08-11T16:10:25.674Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:10:25.755Z [qtp2045766957-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=562a531b-caba-429b-b9d2-d26d76ecfc07 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:10:25.755014Z InvocationID=e7415bed-1410-4c69-a61b-864c9da79381 LogTimestamp=2021-08-11T16:10:25.755083Z Entering 2021-08-11T16:10:25.755Z [qtp2045766957-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=562a531b-caba-429b-b9d2-d26d76ecfc07 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:10:25.755014Z InvocationID=e7415bed-1410-4c69-a61b-864c9da79381 LogTimestamp=2021-08-11T16:10:25.755083Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:10:25.756Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=28882ae7-758d-438b-87f2-6f36a7f6e632 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:10:25.756Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=28882ae7-758d-438b-87f2-6f36a7f6e632 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@73d086df 2021-08-11T16:10:25.756Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=28882ae7-758d-438b-87f2-6f36a7f6e632 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:10:25.756Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=28882ae7-758d-438b-87f2-6f36a7f6e632 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:10:26.854Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:25.756Z InvocationID=011023f9-d8f5-42aa-a190-2edb9eeacd68 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:26.854Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:25.756Z InvocationID=011023f9-d8f5-42aa-a190-2edb9eeacd68 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:26.854Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:25.756Z InvocationID=011023f9-d8f5-42aa-a190-2edb9eeacd68 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:26.854Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:25.756Z InvocationID=011023f9-d8f5-42aa-a190-2edb9eeacd68 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:26.854Z [qtp2045766957-11] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:25.756Z InvocationID=011023f9-d8f5-42aa-a190-2edb9eeacd68 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name GenericCr 2021-08-11T16:10:28.960Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f731373a-a4dc-436e-bc41-90527bb21784 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:10:28.960Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f731373a-a4dc-436e-bc41-90527bb21784 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6df35e81 2021-08-11T16:10:28.960Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f731373a-a4dc-436e-bc41-90527bb21784 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:10:28.960Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f731373a-a4dc-436e-bc41-90527bb21784 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:10:29.560Z [qtp2045766957-11] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Generic_CR" RESOURCE_TYPE = "ABSTRACT" SERVICE_INSTANCE_ID = "f798f172-e170-4beb-8332-ce35b75bf7b3" INVARIANT_UUID = "9cf06b5f-68b5-4fd3-9944-394169865870" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.abstract.nodes.CR" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f798f172-e170-4beb-8332-ce35b75bf7b3 EntryTimestamp=2021-08-11T16:10:25.756Z InvocationID=011023f9-d8f5-42aa-a190-2edb9eeacd68 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id 33d72586-295d-4575-bc44-90297e500928 2021-08-11T16:10:31.176Z [qtp2045766957-11] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Generic_CR" RESOURCE_TYPE = "ABSTRACT" SERVICE_INSTANCE_ID = "f798f172-e170-4beb-8332-ce35b75bf7b3" INVARIANT_UUID = "9cf06b5f-68b5-4fd3-9944-394169865870" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "33d72586-295d-4575-bc44-90297e500928.assettoscatemplate" CURR_ARTIFACT_UUID = "33d72586-295d-4575-bc44-90297e500928.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-GenericCr-template.yml,0,33d72586-295d-4575-bc44-90297e500928.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=5420 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=da1678b2-0500-4e24-8be8-6620fe341bf7 EntryTimestamp=2021-08-11T16:10:25.756Z InvocationID=011023f9-d8f5-42aa-a190-2edb9eeacd68 LogTimestamp=2021-08-11T16:10:31.176476Z ResponseDesc=Created Exiting. 2021-08-11T16:10:31.177Z [qtp2045766957-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:10:31.189Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=6c71b41b-3cf2-455d-b178-313fbbc7e7df InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:10:31.189386Z InvocationID=cac62029-b0af-410a-ae42-5b214363ef80 LogTimestamp=2021-08-11T16:10:31.189437Z Entering 2021-08-11T16:10:31.189Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=6c71b41b-3cf2-455d-b178-313fbbc7e7df InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:10:31.189386Z InvocationID=cac62029-b0af-410a-ae42-5b214363ef80 LogTimestamp=2021-08-11T16:10:31.189437Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:10:31.440Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:31.190Z InvocationID=47d984c2-e27f-4ceb-b01e-066b3f13f8fc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:31.440Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:31.190Z InvocationID=47d984c2-e27f-4ceb-b01e-066b3f13f8fc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:31.441Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:31.190Z InvocationID=47d984c2-e27f-4ceb-b01e-066b3f13f8fc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:31.441Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:31.190Z InvocationID=47d984c2-e27f-4ceb-b01e-066b3f13f8fc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:31.441Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:31.190Z InvocationID=47d984c2-e27f-4ceb-b01e-066b3f13f8fc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name GenericPnf 2021-08-11T16:10:31.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ba8cdae-8e3e-4a6c-a050-afea62e4d1ab PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:10:31.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ba8cdae-8e3e-4a6c-a050-afea62e4d1ab PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4c8ede54 2021-08-11T16:10:31.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ba8cdae-8e3e-4a6c-a050-afea62e4d1ab PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:10:31.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ba8cdae-8e3e-4a6c-a050-afea62e4d1ab PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:10:32.969Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Generic_PNF" RESOURCE_TYPE = "ABSTRACT" SERVICE_INSTANCE_ID = "2ee6785b-2bd1-4756-ab79-63b57b788bed" INVARIANT_UUID = "e6634e16-57ad-4271-99cb-3703e0aa3da9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.abstract.nodes.PNF" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2ee6785b-2bd1-4756-ab79-63b57b788bed EntryTimestamp=2021-08-11T16:10:31.190Z InvocationID=47d984c2-e27f-4ceb-b01e-066b3f13f8fc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id ceadb4c2-a1ef-41c8-b9f7-5678da9c1e41 2021-08-11T16:10:33.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:10:33.320Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:10:34.595Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Generic_PNF" RESOURCE_TYPE = "ABSTRACT" SERVICE_INSTANCE_ID = "2ee6785b-2bd1-4756-ab79-63b57b788bed" INVARIANT_UUID = "e6634e16-57ad-4271-99cb-3703e0aa3da9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "ceadb4c2-a1ef-41c8-b9f7-5678da9c1e41.assettoscatemplate" CURR_ARTIFACT_UUID = "ceadb4c2-a1ef-41c8-b9f7-5678da9c1e41.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-GenericPnf-template.yml,0,ceadb4c2-a1ef-41c8-b9f7-5678da9c1e41.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=3404 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=93ac1851-9110-43e2-bf5a-91e9e11ae8b4 EntryTimestamp=2021-08-11T16:10:31.190Z InvocationID=47d984c2-e27f-4ceb-b01e-066b3f13f8fc LogTimestamp=2021-08-11T16:10:34.594356Z ResponseDesc=Created Exiting. 2021-08-11T16:10:34.595Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:10:34.612Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=d08eb5ee-51ee-4619-ab99-64785376984e InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:10:34.612039Z InvocationID=7f0bc539-a07c-4910-84e9-0991d89c3c9d LogTimestamp=2021-08-11T16:10:34.612107Z Entering 2021-08-11T16:10:34.612Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=d08eb5ee-51ee-4619-ab99-64785376984e InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:10:34.612039Z InvocationID=7f0bc539-a07c-4910-84e9-0991d89c3c9d LogTimestamp=2021-08-11T16:10:34.612107Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:10:34.707Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df4cd5c2-50a1-4c1d-9ff4-ea2b4c4b1e8b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:10:34.707Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df4cd5c2-50a1-4c1d-9ff4-ea2b4c4b1e8b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3a725f06 2021-08-11T16:10:34.707Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df4cd5c2-50a1-4c1d-9ff4-ea2b4c4b1e8b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:10:34.708Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df4cd5c2-50a1-4c1d-9ff4-ea2b4c4b1e8b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:10:35.010Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:34.612Z InvocationID=5f155f73-3334-49fc-ad9f-d190dd840e98 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:35.010Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:34.612Z InvocationID=5f155f73-3334-49fc-ad9f-d190dd840e98 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:35.010Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:34.612Z InvocationID=5f155f73-3334-49fc-ad9f-d190dd840e98 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:35.010Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:34.612Z InvocationID=5f155f73-3334-49fc-ad9f-d190dd840e98 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:35.010Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:34.612Z InvocationID=5f155f73-3334-49fc-ad9f-d190dd840e98 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name GenericService 2021-08-11T16:10:36.435Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Generic_Service" RESOURCE_TYPE = "ABSTRACT" SERVICE_INSTANCE_ID = "3296b452-fe1b-4018-8333-4830854d6600" INVARIANT_UUID = "37368edb-9a28-48f8-aab5-a825bcd17d41" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.abstract.nodes.service" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3296b452-fe1b-4018-8333-4830854d6600 EntryTimestamp=2021-08-11T16:10:34.612Z InvocationID=5f155f73-3334-49fc-ad9f-d190dd840e98 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id 6aa2223d-56d2-4a75-9152-cf80635dc83d 2021-08-11T16:10:37.688Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=404712bf-724a-49da-bd3e-e83c039c9194 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:10:37.688Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=404712bf-724a-49da-bd3e-e83c039c9194 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4de3c0e9 2021-08-11T16:10:37.688Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=404712bf-724a-49da-bd3e-e83c039c9194 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:10:37.688Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=404712bf-724a-49da-bd3e-e83c039c9194 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:10:38.090Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Generic_Service" RESOURCE_TYPE = "ABSTRACT" SERVICE_INSTANCE_ID = "3296b452-fe1b-4018-8333-4830854d6600" INVARIANT_UUID = "37368edb-9a28-48f8-aab5-a825bcd17d41" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "6aa2223d-56d2-4a75-9152-cf80635dc83d.assettoscatemplate" CURR_ARTIFACT_UUID = "6aa2223d-56d2-4a75-9152-cf80635dc83d.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-GenericService-template.yml,0,6aa2223d-56d2-4a75-9152-cf80635dc83d.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=3477 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=66a9e2fc-3681-4bb1-b1da-13e2904b46d5 EntryTimestamp=2021-08-11T16:10:34.612Z InvocationID=5f155f73-3334-49fc-ad9f-d190dd840e98 LogTimestamp=2021-08-11T16:10:38.089692Z ResponseDesc=Created Exiting. 2021-08-11T16:10:38.091Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:10:38.108Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=f35f52ed-30e6-4cbc-bc10-5ab5bceb3f8c InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:10:38.107943Z InvocationID=624e11b4-bfb6-4c1d-b816-53a2591e296d LogTimestamp=2021-08-11T16:10:38.108002Z Entering 2021-08-11T16:10:38.108Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=f35f52ed-30e6-4cbc-bc10-5ab5bceb3f8c InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:10:38.107943Z InvocationID=624e11b4-bfb6-4c1d-b816-53a2591e296d LogTimestamp=2021-08-11T16:10:38.108002Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:10:38.437Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:38.108Z InvocationID=079142fd-a3ed-4691-9254-99acd6f7fff5 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:38.438Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:38.108Z InvocationID=079142fd-a3ed-4691-9254-99acd6f7fff5 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:38.438Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:38.108Z InvocationID=079142fd-a3ed-4691-9254-99acd6f7fff5 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:38.438Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:38.108Z InvocationID=079142fd-a3ed-4691-9254-99acd6f7fff5 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:38.450Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:38.108Z InvocationID=079142fd-a3ed-4691-9254-99acd6f7fff5 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Contrailnetworkrules 2021-08-11T16:10:39.712Z [qtp2045766957-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailNetworkRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "528e2076-2657-476f-93ec-3fa0b8ab2e62" INVARIANT_UUID = "c1994000-2203-4be0-8901-d896fa9e5c49" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.rules.nodes.heat.network.contrail.NetworkRules" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=528e2076-2657-476f-93ec-3fa0b8ab2e62 EntryTimestamp=2021-08-11T16:10:38.108Z InvocationID=079142fd-a3ed-4691-9254-99acd6f7fff5 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:10:39.713Z [qtp2045766957-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailNetworkRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "528e2076-2657-476f-93ec-3fa0b8ab2e62" INVARIANT_UUID = "c1994000-2203-4be0-8901-d896fa9e5c49" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.rules.nodes.heat.network.contrail.NetworkRules" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=528e2076-2657-476f-93ec-3fa0b8ab2e62 TargetServiceName=null EntryTimestamp=2021-08-11T16:10:38.108Z InvocationID=079142fd-a3ed-4691-9254-99acd6f7fff5 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:10:39.961Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailNetworkRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "528e2076-2657-476f-93ec-3fa0b8ab2e62" INVARIANT_UUID = "c1994000-2203-4be0-8901-d896fa9e5c49" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.rules.nodes.heat.network.contrail.NetworkRules" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=528e2076-2657-476f-93ec-3fa0b8ab2e62 TargetServiceName=null EntryTimestamp=2021-08-11T16:10:38.108Z InvocationID=079142fd-a3ed-4691-9254-99acd6f7fff5 ErrorCode=500 lock resource with id e0f53d09-6003-431c-9911-6e557007f691 2021-08-11T16:10:40.695Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a362315e-7cb4-4fef-afca-930cd72d0fa8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:10:40.695Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a362315e-7cb4-4fef-afca-930cd72d0fa8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3a139882 2021-08-11T16:10:40.695Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a362315e-7cb4-4fef-afca-930cd72d0fa8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:10:40.696Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a362315e-7cb4-4fef-afca-930cd72d0fa8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:10:41.561Z [qtp2045766957-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailNetworkRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "528e2076-2657-476f-93ec-3fa0b8ab2e62" INVARIANT_UUID = "c1994000-2203-4be0-8901-d896fa9e5c49" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "e0f53d09-6003-431c-9911-6e557007f691.assettoscatemplate" CURR_ARTIFACT_UUID = "e0f53d09-6003-431c-9911-6e557007f691.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailnetworkrules-template.yml,0,e0f53d09-6003-431c-9911-6e557007f691.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9408f367-9e8e-48a7-8308-8fd8b754497f TargetServiceName=null EntryTimestamp=2021-08-11T16:10:38.108Z InvocationID=079142fd-a3ed-4691-9254-99acd6f7fff5 ErrorCode=500 Starting to create Csar for component ContrailNetworkRules 2021-08-11T16:10:41.761Z [qtp2045766957-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailNetworkRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "528e2076-2657-476f-93ec-3fa0b8ab2e62" INVARIANT_UUID = "c1994000-2203-4be0-8901-d896fa9e5c49" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "e0f53d09-6003-431c-9911-6e557007f691.assettoscatemplate" CURR_ARTIFACT_UUID = "e0f53d09-6003-431c-9911-6e557007f691.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailnetworkrules-template.yml,0,e0f53d09-6003-431c-9911-6e557007f691.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9408f367-9e8e-48a7-8308-8fd8b754497f TargetServiceName=null EntryTimestamp=2021-08-11T16:10:38.108Z InvocationID=079142fd-a3ed-4691-9254-99acd6f7fff5 ErrorCode=500 Ended create Csar for component ContrailNetworkRules 2021-08-11T16:10:41.762Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailNetworkRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "528e2076-2657-476f-93ec-3fa0b8ab2e62" INVARIANT_UUID = "c1994000-2203-4be0-8901-d896fa9e5c49" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "e0f53d09-6003-431c-9911-6e557007f691.assettoscatemplate" CURR_ARTIFACT_UUID = "e0f53d09-6003-431c-9911-6e557007f691.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailnetworkrules-template.yml,0,e0f53d09-6003-431c-9911-6e557007f691.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9408f367-9e8e-48a7-8308-8fd8b754497f TargetServiceName=null EntryTimestamp=2021-08-11T16:10:38.108Z InvocationID=079142fd-a3ed-4691-9254-99acd6f7fff5 ErrorCode=500 lock resource with id e0f53d09-6003-431c-9911-6e557007f691 2021-08-11T16:10:43.384Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailNetworkRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "528e2076-2657-476f-93ec-3fa0b8ab2e62" INVARIANT_UUID = "c1994000-2203-4be0-8901-d896fa9e5c49" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "e0f53d09-6003-431c-9911-6e557007f691.assettoscacsar" CURR_ARTIFACT_UUID = "e0f53d09-6003-431c-9911-6e557007f691.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Contrailnetworkrules-csar.csar,0,e0f53d09-6003-431c-9911-6e557007f691.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5276 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9eaa0e8b-1ae5-4cc7-a20a-ad98a58cb4bb TargetServiceName=null EntryTimestamp=2021-08-11T16:10:38.108Z InvocationID=079142fd-a3ed-4691-9254-99acd6f7fff5 ErrorCode=500 LogTimestamp=2021-08-11T16:10:43.384641Z ResponseDesc=Created Exiting. 2021-08-11T16:10:43.385Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:10:43.403Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=2b8918b8-d211-411c-b15f-3286513aa6db InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:10:43.403522Z InvocationID=bdd127d9-688d-4f6d-ad2b-e84fa89cd859 LogTimestamp=2021-08-11T16:10:43.403575Z Entering 2021-08-11T16:10:43.403Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=2b8918b8-d211-411c-b15f-3286513aa6db InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:10:43.403522Z InvocationID=bdd127d9-688d-4f6d-ad2b-e84fa89cd859 LogTimestamp=2021-08-11T16:10:43.403575Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:10:43.699Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c5dc866-dfdf-429e-b68c-4dbdbf4c5c91 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:10:43.699Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c5dc866-dfdf-429e-b68c-4dbdbf4c5c91 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@558dea 2021-08-11T16:10:43.699Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c5dc866-dfdf-429e-b68c-4dbdbf4c5c91 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:10:43.699Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c5dc866-dfdf-429e-b68c-4dbdbf4c5c91 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:10:43.870Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:43.404Z InvocationID=f7f94baf-142b-43db-994e-7e28e9594f6e PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:43.870Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:43.404Z InvocationID=f7f94baf-142b-43db-994e-7e28e9594f6e PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:43.870Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:43.404Z InvocationID=f7f94baf-142b-43db-994e-7e28e9594f6e PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:43.871Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:43.404Z InvocationID=f7f94baf-142b-43db-994e-7e28e9594f6e PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:43.871Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:43.404Z InvocationID=f7f94baf-142b-43db-994e-7e28e9594f6e PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Contrailport 2021-08-11T16:10:45.151Z [qtp2045766957-14] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailPort" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "d3c32abb-d6ae-430a-995e-672eb81a9453" INVARIANT_UUID = "21c2d88d-662c-47b4-aece-df68e52d264f" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.heat.network.contrail.Port" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d3c32abb-d6ae-430a-995e-672eb81a9453 EntryTimestamp=2021-08-11T16:10:43.404Z InvocationID=f7f94baf-142b-43db-994e-7e28e9594f6e PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:10:45.152Z [qtp2045766957-14] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailPort" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "d3c32abb-d6ae-430a-995e-672eb81a9453" INVARIANT_UUID = "21c2d88d-662c-47b4-aece-df68e52d264f" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.heat.network.contrail.Port" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d3c32abb-d6ae-430a-995e-672eb81a9453 TargetServiceName=null EntryTimestamp=2021-08-11T16:10:43.404Z InvocationID=f7f94baf-142b-43db-994e-7e28e9594f6e ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:10:45.362Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailPort" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "d3c32abb-d6ae-430a-995e-672eb81a9453" INVARIANT_UUID = "21c2d88d-662c-47b4-aece-df68e52d264f" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.heat.network.contrail.Port" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d3c32abb-d6ae-430a-995e-672eb81a9453 TargetServiceName=null EntryTimestamp=2021-08-11T16:10:43.404Z InvocationID=f7f94baf-142b-43db-994e-7e28e9594f6e ErrorCode=500 lock resource with id 72d6faaa-dfe1-46b0-965c-5f8536fe164c 2021-08-11T16:10:46.608Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8a16aa6-1d62-40d7-a7b1-8e9d14e3f7f6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:10:46.608Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8a16aa6-1d62-40d7-a7b1-8e9d14e3f7f6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@28777889 2021-08-11T16:10:46.608Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8a16aa6-1d62-40d7-a7b1-8e9d14e3f7f6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:10:46.609Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8a16aa6-1d62-40d7-a7b1-8e9d14e3f7f6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:10:46.965Z [qtp2045766957-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailPort" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "d3c32abb-d6ae-430a-995e-672eb81a9453" INVARIANT_UUID = "21c2d88d-662c-47b4-aece-df68e52d264f" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "72d6faaa-dfe1-46b0-965c-5f8536fe164c.assettoscatemplate" CURR_ARTIFACT_UUID = "72d6faaa-dfe1-46b0-965c-5f8536fe164c.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailport-template.yml,0,72d6faaa-dfe1-46b0-965c-5f8536fe164c.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b098e62e-d5bb-4710-9600-d946ea317783 TargetServiceName=null EntryTimestamp=2021-08-11T16:10:43.404Z InvocationID=f7f94baf-142b-43db-994e-7e28e9594f6e ErrorCode=500 Starting to create Csar for component ContrailPort 2021-08-11T16:10:47.143Z [qtp2045766957-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailPort" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "d3c32abb-d6ae-430a-995e-672eb81a9453" INVARIANT_UUID = "21c2d88d-662c-47b4-aece-df68e52d264f" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "72d6faaa-dfe1-46b0-965c-5f8536fe164c.assettoscatemplate" CURR_ARTIFACT_UUID = "72d6faaa-dfe1-46b0-965c-5f8536fe164c.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailport-template.yml,0,72d6faaa-dfe1-46b0-965c-5f8536fe164c.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b098e62e-d5bb-4710-9600-d946ea317783 TargetServiceName=null EntryTimestamp=2021-08-11T16:10:43.404Z InvocationID=f7f94baf-142b-43db-994e-7e28e9594f6e ErrorCode=500 Ended create Csar for component ContrailPort 2021-08-11T16:10:47.143Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailPort" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "d3c32abb-d6ae-430a-995e-672eb81a9453" INVARIANT_UUID = "21c2d88d-662c-47b4-aece-df68e52d264f" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "72d6faaa-dfe1-46b0-965c-5f8536fe164c.assettoscatemplate" CURR_ARTIFACT_UUID = "72d6faaa-dfe1-46b0-965c-5f8536fe164c.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailport-template.yml,0,72d6faaa-dfe1-46b0-965c-5f8536fe164c.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b098e62e-d5bb-4710-9600-d946ea317783 TargetServiceName=null EntryTimestamp=2021-08-11T16:10:43.404Z InvocationID=f7f94baf-142b-43db-994e-7e28e9594f6e ErrorCode=500 lock resource with id 72d6faaa-dfe1-46b0-965c-5f8536fe164c 2021-08-11T16:10:48.305Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:10:48.311Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:10:48.967Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailPort" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "d3c32abb-d6ae-430a-995e-672eb81a9453" INVARIANT_UUID = "21c2d88d-662c-47b4-aece-df68e52d264f" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "72d6faaa-dfe1-46b0-965c-5f8536fe164c.assettoscacsar" CURR_ARTIFACT_UUID = "72d6faaa-dfe1-46b0-965c-5f8536fe164c.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Contrailport-csar.csar,0,72d6faaa-dfe1-46b0-965c-5f8536fe164c.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5563 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7032e21e-1c63-430c-a5af-31158d34d62f TargetServiceName=null EntryTimestamp=2021-08-11T16:10:43.404Z InvocationID=f7f94baf-142b-43db-994e-7e28e9594f6e ErrorCode=500 LogTimestamp=2021-08-11T16:10:48.967297Z ResponseDesc=Created Exiting. 2021-08-11T16:10:48.968Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:10:49.053Z [qtp2045766957-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=ffe483a8-0e65-4d97-b7d5-febc0f908dcc InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:10:49.052803Z InvocationID=3e5e0ca3-6a2c-40c5-a43d-a1558cb5d6fc LogTimestamp=2021-08-11T16:10:49.052881Z Entering 2021-08-11T16:10:49.053Z [qtp2045766957-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=ffe483a8-0e65-4d97-b7d5-febc0f908dcc InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:10:49.052803Z InvocationID=3e5e0ca3-6a2c-40c5-a43d-a1558cb5d6fc LogTimestamp=2021-08-11T16:10:49.052881Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:10:49.290Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:10:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:10:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:10:50.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d723bec-c1fa-45a6-9d74-db55b807ca68 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:10:50.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d723bec-c1fa-45a6-9d74-db55b807ca68 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7bc011e6 2021-08-11T16:10:50.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d723bec-c1fa-45a6-9d74-db55b807ca68 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:10:50.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d723bec-c1fa-45a6-9d74-db55b807ca68 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:10:51.174Z [qtp2045766957-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:49.054Z InvocationID=7dcd9edf-08e6-43fa-b916-04870bed94bc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:51.174Z [qtp2045766957-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:49.054Z InvocationID=7dcd9edf-08e6-43fa-b916-04870bed94bc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:51.174Z [qtp2045766957-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:49.054Z InvocationID=7dcd9edf-08e6-43fa-b916-04870bed94bc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:51.174Z [qtp2045766957-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:49.054Z InvocationID=7dcd9edf-08e6-43fa-b916-04870bed94bc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:10:51.362Z [qtp2045766957-20] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:10:49.054Z InvocationID=7dcd9edf-08e6-43fa-b916-04870bed94bc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name PortMirroringConfiguration 2021-08-11T16:10:53.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4bb748b-9ddf-4a2d-a0f4-d54b740bd745 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:10:53.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4bb748b-9ddf-4a2d-a0f4-d54b740bd745 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5d314194 2021-08-11T16:10:53.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4bb748b-9ddf-4a2d-a0f4-d54b740bd745 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:10:53.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4bb748b-9ddf-4a2d-a0f4-d54b740bd745 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:10:56.554Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba9a9009-2afc-452b-8e9b-70de30c9e431 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:10:56.554Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba9a9009-2afc-452b-8e9b-70de30c9e431 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7127ad05 2021-08-11T16:10:56.554Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba9a9009-2afc-452b-8e9b-70de30c9e431 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:10:56.555Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba9a9009-2afc-452b-8e9b-70de30c9e431 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:10:57.549Z [qtp2045766957-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Port Mirroring Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "235d4843-ce19-4cae-a0a3-a743ca0ebe2e" INVARIANT_UUID = "9e79b4c8-b0fc-434e-b18c-eb96b5ac4768" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.nodes.PortMirroringConfiguration" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=235d4843-ce19-4cae-a0a3-a743ca0ebe2e EntryTimestamp=2021-08-11T16:10:49.054Z InvocationID=7dcd9edf-08e6-43fa-b916-04870bed94bc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id 0fef8bb4-5843-4439-be64-2178f5d15cfe 2021-08-11T16:10:59.665Z [qtp2045766957-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Port Mirroring Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "235d4843-ce19-4cae-a0a3-a743ca0ebe2e" INVARIANT_UUID = "9e79b4c8-b0fc-434e-b18c-eb96b5ac4768" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "0fef8bb4-5843-4439-be64-2178f5d15cfe.assettoscatemplate" CURR_ARTIFACT_UUID = "0fef8bb4-5843-4439-be64-2178f5d15cfe.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-PortMirroringConfiguration-template.yml,0,0fef8bb4-5843-4439-be64-2178f5d15cfe.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7458b87e-5548-469b-bb74-a0be060492a8 EntryTimestamp=2021-08-11T16:10:49.054Z InvocationID=7dcd9edf-08e6-43fa-b916-04870bed94bc PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Starting to create Csar for component Port Mirroring Configuration 2021-08-11T16:10:59.874Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7319e6bd-b13c-4459-ab74-643666883e76 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:10:59.874Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7319e6bd-b13c-4459-ab74-643666883e76 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@62072098 2021-08-11T16:10:59.874Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7319e6bd-b13c-4459-ab74-643666883e76 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:10:59.875Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7319e6bd-b13c-4459-ab74-643666883e76 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:10:59.936Z [qtp2045766957-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Port Mirroring Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "235d4843-ce19-4cae-a0a3-a743ca0ebe2e" INVARIANT_UUID = "9e79b4c8-b0fc-434e-b18c-eb96b5ac4768" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "0fef8bb4-5843-4439-be64-2178f5d15cfe.assettoscatemplate" CURR_ARTIFACT_UUID = "0fef8bb4-5843-4439-be64-2178f5d15cfe.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-PortMirroringConfiguration-template.yml,0,0fef8bb4-5843-4439-be64-2178f5d15cfe.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7458b87e-5548-469b-bb74-a0be060492a8 EntryTimestamp=2021-08-11T16:10:49.054Z InvocationID=7dcd9edf-08e6-43fa-b916-04870bed94bc Ended create Csar for component Port Mirroring Configuration 2021-08-11T16:10:59.936Z [qtp2045766957-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Port Mirroring Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "235d4843-ce19-4cae-a0a3-a743ca0ebe2e" INVARIANT_UUID = "9e79b4c8-b0fc-434e-b18c-eb96b5ac4768" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "0fef8bb4-5843-4439-be64-2178f5d15cfe.assettoscatemplate" CURR_ARTIFACT_UUID = "0fef8bb4-5843-4439-be64-2178f5d15cfe.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-PortMirroringConfiguration-template.yml,0,0fef8bb4-5843-4439-be64-2178f5d15cfe.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7458b87e-5548-469b-bb74-a0be060492a8 EntryTimestamp=2021-08-11T16:10:49.054Z InvocationID=7dcd9edf-08e6-43fa-b916-04870bed94bc lock resource with id 0fef8bb4-5843-4439-be64-2178f5d15cfe 2021-08-11T16:11:01.883Z [qtp2045766957-20] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Port Mirroring Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "235d4843-ce19-4cae-a0a3-a743ca0ebe2e" INVARIANT_UUID = "9e79b4c8-b0fc-434e-b18c-eb96b5ac4768" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "0fef8bb4-5843-4439-be64-2178f5d15cfe.assettoscacsar" CURR_ARTIFACT_UUID = "0fef8bb4-5843-4439-be64-2178f5d15cfe.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-PortMirroringConfiguration-csar.csar,0,0fef8bb4-5843-4439-be64-2178f5d15cfe.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=12829 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b2fb5e82-05fd-42d6-81b0-859e1691c097 EntryTimestamp=2021-08-11T16:10:49.054Z InvocationID=7dcd9edf-08e6-43fa-b916-04870bed94bc LogTimestamp=2021-08-11T16:11:01.883190Z ResponseDesc=Created Exiting. 2021-08-11T16:11:01.884Z [qtp2045766957-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:11:01.970Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=f6014ba5-c94e-4787-97df-a72736803dfc InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:11:01.969620Z InvocationID=2bf298e3-811f-448a-a311-c99b196cc6c4 LogTimestamp=2021-08-11T16:11:01.969697Z Entering 2021-08-11T16:11:01.970Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=f6014ba5-c94e-4787-97df-a72736803dfc InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:11:01.969620Z InvocationID=2bf298e3-811f-448a-a311-c99b196cc6c4 LogTimestamp=2021-08-11T16:11:01.969697Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:11:02.273Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b16f0619-495b-46fe-bad0-a3c5a38352d2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:11:02.273Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b16f0619-495b-46fe-bad0-a3c5a38352d2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1933521f 2021-08-11T16:11:02.273Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b16f0619-495b-46fe-bad0-a3c5a38352d2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:11:02.273Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b16f0619-495b-46fe-bad0-a3c5a38352d2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:11:03.180Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:01.970Z InvocationID=fa86c846-1871-4571-bc13-ebeab4cbab74 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:03.180Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:01.970Z InvocationID=fa86c846-1871-4571-bc13-ebeab4cbab74 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:03.180Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:01.970Z InvocationID=fa86c846-1871-4571-bc13-ebeab4cbab74 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:03.180Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:01.970Z InvocationID=fa86c846-1871-4571-bc13-ebeab4cbab74 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:03.181Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:01.970Z InvocationID=fa86c846-1871-4571-bc13-ebeab4cbab74 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Serviceproxy 2021-08-11T16:11:03.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:11:03.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:11:04.261Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:11:04.813Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d57609b3-fe69-4f18-a786-1f93f9dfb311 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:11:04.813Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d57609b3-fe69-4f18-a786-1f93f9dfb311 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4ed99d4c 2021-08-11T16:11:04.814Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d57609b3-fe69-4f18-a786-1f93f9dfb311 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:11:04.814Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d57609b3-fe69-4f18-a786-1f93f9dfb311 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:11:05.057Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "serviceProxy" RESOURCE_TYPE = "ServiceProxy" SERVICE_INSTANCE_ID = "c13d3fb5-6bb2-4e43-ae28-fe6da4fd8fad" INVARIANT_UUID = "c20c548b-2afc-4d0b-b32d-0ad8d6482791" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.nodes.ServiceProxy" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c13d3fb5-6bb2-4e43-ae28-fe6da4fd8fad EntryTimestamp=2021-08-11T16:11:01.970Z InvocationID=fa86c846-1871-4571-bc13-ebeab4cbab74 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id 2e1cf145-b85f-4581-8c3f-8b6cdc427168 2021-08-11T16:11:06.674Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "serviceProxy" RESOURCE_TYPE = "ServiceProxy" SERVICE_INSTANCE_ID = "c13d3fb5-6bb2-4e43-ae28-fe6da4fd8fad" INVARIANT_UUID = "c20c548b-2afc-4d0b-b32d-0ad8d6482791" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "2e1cf145-b85f-4581-8c3f-8b6cdc427168.assettoscatemplate" CURR_ARTIFACT_UUID = "2e1cf145-b85f-4581-8c3f-8b6cdc427168.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Serviceproxy-template.yml,0,2e1cf145-b85f-4581-8c3f-8b6cdc427168.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=4703 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=198a1e5c-9465-45c9-8732-8da156d5d203 EntryTimestamp=2021-08-11T16:11:01.970Z InvocationID=fa86c846-1871-4571-bc13-ebeab4cbab74 LogTimestamp=2021-08-11T16:11:06.673992Z ResponseDesc=Created Exiting. 2021-08-11T16:11:06.675Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:11:06.690Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=03cadaf8-9dc8-4c19-8157-2f7fa71e7b14 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:11:06.689810Z InvocationID=796053aa-649b-492d-b970-0d12d17b669b LogTimestamp=2021-08-11T16:11:06.689858Z Entering 2021-08-11T16:11:06.690Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=03cadaf8-9dc8-4c19-8157-2f7fa71e7b14 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:11:06.689810Z InvocationID=796053aa-649b-492d-b970-0d12d17b669b LogTimestamp=2021-08-11T16:11:06.689858Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:11:07.012Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:06.690Z InvocationID=cf5c0b08-0d74-4930-a3e9-e11b5e1ba9f6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:07.012Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:06.690Z InvocationID=cf5c0b08-0d74-4930-a3e9-e11b5e1ba9f6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:07.013Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:06.690Z InvocationID=cf5c0b08-0d74-4930-a3e9-e11b5e1ba9f6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:07.013Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:06.690Z InvocationID=cf5c0b08-0d74-4930-a3e9-e11b5e1ba9f6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:07.029Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:06.690Z InvocationID=cf5c0b08-0d74-4930-a3e9-e11b5e1ba9f6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Contrailv2networkrules 2021-08-11T16:11:07.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c9e5ac28-40d0-49f7-844f-296174a70a4b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:11:07.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c9e5ac28-40d0-49f7-844f-296174a70a4b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2da5e17 2021-08-11T16:11:07.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c9e5ac28-40d0-49f7-844f-296174a70a4b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:11:07.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c9e5ac28-40d0-49f7-844f-296174a70a4b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:11:08.473Z [qtp2045766957-12] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailV2NetworkRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "73850725-c47f-4163-969c-18c776e1e0a5" INVARIANT_UUID = "c498a290-2a84-4cb2-9d52-eba17d864175" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.rules.nodes.heat.network.contrailV2.NetworkRules" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=73850725-c47f-4163-969c-18c776e1e0a5 EntryTimestamp=2021-08-11T16:11:06.690Z InvocationID=cf5c0b08-0d74-4930-a3e9-e11b5e1ba9f6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:11:08.474Z [qtp2045766957-12] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailV2NetworkRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "73850725-c47f-4163-969c-18c776e1e0a5" INVARIANT_UUID = "c498a290-2a84-4cb2-9d52-eba17d864175" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.rules.nodes.heat.network.contrailV2.NetworkRules" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=73850725-c47f-4163-969c-18c776e1e0a5 TargetServiceName=null EntryTimestamp=2021-08-11T16:11:06.690Z InvocationID=cf5c0b08-0d74-4930-a3e9-e11b5e1ba9f6 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:11:08.807Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailV2NetworkRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "73850725-c47f-4163-969c-18c776e1e0a5" INVARIANT_UUID = "c498a290-2a84-4cb2-9d52-eba17d864175" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.rules.nodes.heat.network.contrailV2.NetworkRules" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=73850725-c47f-4163-969c-18c776e1e0a5 TargetServiceName=null EntryTimestamp=2021-08-11T16:11:06.690Z InvocationID=cf5c0b08-0d74-4930-a3e9-e11b5e1ba9f6 ErrorCode=500 lock resource with id 3d3a6e87-2bca-4201-8658-abc8b6a6e231 2021-08-11T16:11:10.421Z [qtp2045766957-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailV2NetworkRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "73850725-c47f-4163-969c-18c776e1e0a5" INVARIANT_UUID = "c498a290-2a84-4cb2-9d52-eba17d864175" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "3d3a6e87-2bca-4201-8658-abc8b6a6e231.assettoscatemplate" CURR_ARTIFACT_UUID = "3d3a6e87-2bca-4201-8658-abc8b6a6e231.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailv2networkrules-template.yml,0,3d3a6e87-2bca-4201-8658-abc8b6a6e231.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b11413f8-102a-4cf0-9530-5e4948ad52c0 TargetServiceName=null EntryTimestamp=2021-08-11T16:11:06.690Z InvocationID=cf5c0b08-0d74-4930-a3e9-e11b5e1ba9f6 ErrorCode=500 Starting to create Csar for component ContrailV2NetworkRules 2021-08-11T16:11:10.595Z [qtp2045766957-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailV2NetworkRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "73850725-c47f-4163-969c-18c776e1e0a5" INVARIANT_UUID = "c498a290-2a84-4cb2-9d52-eba17d864175" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "3d3a6e87-2bca-4201-8658-abc8b6a6e231.assettoscatemplate" CURR_ARTIFACT_UUID = "3d3a6e87-2bca-4201-8658-abc8b6a6e231.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailv2networkrules-template.yml,0,3d3a6e87-2bca-4201-8658-abc8b6a6e231.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b11413f8-102a-4cf0-9530-5e4948ad52c0 TargetServiceName=null EntryTimestamp=2021-08-11T16:11:06.690Z InvocationID=cf5c0b08-0d74-4930-a3e9-e11b5e1ba9f6 ErrorCode=500 Ended create Csar for component ContrailV2NetworkRules 2021-08-11T16:11:10.596Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailV2NetworkRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "73850725-c47f-4163-969c-18c776e1e0a5" INVARIANT_UUID = "c498a290-2a84-4cb2-9d52-eba17d864175" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "3d3a6e87-2bca-4201-8658-abc8b6a6e231.assettoscatemplate" CURR_ARTIFACT_UUID = "3d3a6e87-2bca-4201-8658-abc8b6a6e231.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailv2networkrules-template.yml,0,3d3a6e87-2bca-4201-8658-abc8b6a6e231.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b11413f8-102a-4cf0-9530-5e4948ad52c0 TargetServiceName=null EntryTimestamp=2021-08-11T16:11:06.690Z InvocationID=cf5c0b08-0d74-4930-a3e9-e11b5e1ba9f6 ErrorCode=500 lock resource with id 3d3a6e87-2bca-4201-8658-abc8b6a6e231 2021-08-11T16:11:10.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=afe589a2-af4f-4456-b374-8a0bb26aac7a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:11:10.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=afe589a2-af4f-4456-b374-8a0bb26aac7a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5902bd34 2021-08-11T16:11:10.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=afe589a2-af4f-4456-b374-8a0bb26aac7a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:11:10.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=afe589a2-af4f-4456-b374-8a0bb26aac7a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:11:12.216Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailV2NetworkRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "73850725-c47f-4163-969c-18c776e1e0a5" INVARIANT_UUID = "c498a290-2a84-4cb2-9d52-eba17d864175" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "3d3a6e87-2bca-4201-8658-abc8b6a6e231.assettoscacsar" CURR_ARTIFACT_UUID = "3d3a6e87-2bca-4201-8658-abc8b6a6e231.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Contrailv2networkrules-csar.csar,0,3d3a6e87-2bca-4201-8658-abc8b6a6e231.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5526 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a3773f27-04b2-448b-b161-8765759ae2a0 TargetServiceName=null EntryTimestamp=2021-08-11T16:11:06.690Z InvocationID=cf5c0b08-0d74-4930-a3e9-e11b5e1ba9f6 ErrorCode=500 LogTimestamp=2021-08-11T16:11:12.216262Z ResponseDesc=Created Exiting. 2021-08-11T16:11:12.217Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:11:12.254Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=d82b42f0-1fce-49e7-a735-e6876ba014b7 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:11:12.253912Z InvocationID=228dfa2a-8926-4543-8b63-10a85256fd50 LogTimestamp=2021-08-11T16:11:12.253988Z Entering 2021-08-11T16:11:12.254Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=d82b42f0-1fce-49e7-a735-e6876ba014b7 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:11:12.253912Z InvocationID=228dfa2a-8926-4543-8b63-10a85256fd50 LogTimestamp=2021-08-11T16:11:12.253988Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:11:12.559Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:12.254Z InvocationID=7c231fbe-9636-4a51-9f7c-2c2f47f4df75 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:12.559Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:12.254Z InvocationID=7c231fbe-9636-4a51-9f7c-2c2f47f4df75 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:12.559Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:12.254Z InvocationID=7c231fbe-9636-4a51-9f7c-2c2f47f4df75 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:12.559Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:12.254Z InvocationID=7c231fbe-9636-4a51-9f7c-2c2f47f4df75 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:12.571Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:12.254Z InvocationID=7c231fbe-9636-4a51-9f7c-2c2f47f4df75 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Contrailv2virtualnetwork 2021-08-11T16:11:13.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5823ae57-27fa-4f52-8ea7-b5f35d822c1a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:11:13.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5823ae57-27fa-4f52-8ea7-b5f35d822c1a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5aa2afa1 2021-08-11T16:11:13.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5823ae57-27fa-4f52-8ea7-b5f35d822c1a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:11:13.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5823ae57-27fa-4f52-8ea7-b5f35d822c1a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:11:13.834Z [qtp2045766957-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailV2VirtualNetwork" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "97d8ab0d-e495-4f5b-8d16-f646685b3f9d" INVARIANT_UUID = "c9dc8a15-620c-447a-9cc1-877bc3ac9276" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.nodes.heat.network.contrailV2.VirtualNetwork" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=97d8ab0d-e495-4f5b-8d16-f646685b3f9d EntryTimestamp=2021-08-11T16:11:12.254Z InvocationID=7c231fbe-9636-4a51-9f7c-2c2f47f4df75 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:11:13.835Z [qtp2045766957-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailV2VirtualNetwork" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "97d8ab0d-e495-4f5b-8d16-f646685b3f9d" INVARIANT_UUID = "c9dc8a15-620c-447a-9cc1-877bc3ac9276" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.nodes.heat.network.contrailV2.VirtualNetwork" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=97d8ab0d-e495-4f5b-8d16-f646685b3f9d TargetServiceName=null EntryTimestamp=2021-08-11T16:11:12.254Z InvocationID=7c231fbe-9636-4a51-9f7c-2c2f47f4df75 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:11:14.042Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailV2VirtualNetwork" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "97d8ab0d-e495-4f5b-8d16-f646685b3f9d" INVARIANT_UUID = "c9dc8a15-620c-447a-9cc1-877bc3ac9276" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.nodes.heat.network.contrailV2.VirtualNetwork" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=97d8ab0d-e495-4f5b-8d16-f646685b3f9d TargetServiceName=null EntryTimestamp=2021-08-11T16:11:12.254Z InvocationID=7c231fbe-9636-4a51-9f7c-2c2f47f4df75 ErrorCode=500 lock resource with id 34b6e09a-b44a-45c3-a3cc-219c8bcfc9d4 2021-08-11T16:11:15.656Z [qtp2045766957-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailV2VirtualNetwork" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "97d8ab0d-e495-4f5b-8d16-f646685b3f9d" INVARIANT_UUID = "c9dc8a15-620c-447a-9cc1-877bc3ac9276" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "34b6e09a-b44a-45c3-a3cc-219c8bcfc9d4.assettoscatemplate" CURR_ARTIFACT_UUID = "34b6e09a-b44a-45c3-a3cc-219c8bcfc9d4.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailv2virtualnetwork-template.yml,0,34b6e09a-b44a-45c3-a3cc-219c8bcfc9d4.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9aa249fb-58eb-4e75-aab1-00faad11da54 TargetServiceName=null EntryTimestamp=2021-08-11T16:11:12.254Z InvocationID=7c231fbe-9636-4a51-9f7c-2c2f47f4df75 ErrorCode=500 Starting to create Csar for component ContrailV2VirtualNetwork 2021-08-11T16:11:15.809Z [qtp2045766957-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailV2VirtualNetwork" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "97d8ab0d-e495-4f5b-8d16-f646685b3f9d" INVARIANT_UUID = "c9dc8a15-620c-447a-9cc1-877bc3ac9276" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "34b6e09a-b44a-45c3-a3cc-219c8bcfc9d4.assettoscatemplate" CURR_ARTIFACT_UUID = "34b6e09a-b44a-45c3-a3cc-219c8bcfc9d4.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailv2virtualnetwork-template.yml,0,34b6e09a-b44a-45c3-a3cc-219c8bcfc9d4.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9aa249fb-58eb-4e75-aab1-00faad11da54 TargetServiceName=null EntryTimestamp=2021-08-11T16:11:12.254Z InvocationID=7c231fbe-9636-4a51-9f7c-2c2f47f4df75 ErrorCode=500 Ended create Csar for component ContrailV2VirtualNetwork 2021-08-11T16:11:15.810Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailV2VirtualNetwork" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "97d8ab0d-e495-4f5b-8d16-f646685b3f9d" INVARIANT_UUID = "c9dc8a15-620c-447a-9cc1-877bc3ac9276" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "34b6e09a-b44a-45c3-a3cc-219c8bcfc9d4.assettoscatemplate" CURR_ARTIFACT_UUID = "34b6e09a-b44a-45c3-a3cc-219c8bcfc9d4.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailv2virtualnetwork-template.yml,0,34b6e09a-b44a-45c3-a3cc-219c8bcfc9d4.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9aa249fb-58eb-4e75-aab1-00faad11da54 TargetServiceName=null EntryTimestamp=2021-08-11T16:11:12.254Z InvocationID=7c231fbe-9636-4a51-9f7c-2c2f47f4df75 ErrorCode=500 lock resource with id 34b6e09a-b44a-45c3-a3cc-219c8bcfc9d4 2021-08-11T16:11:16.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94fd46e4-340b-4b91-806b-53db660a0b13 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:11:16.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94fd46e4-340b-4b91-806b-53db660a0b13 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@58137a44 2021-08-11T16:11:16.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94fd46e4-340b-4b91-806b-53db660a0b13 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:11:16.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94fd46e4-340b-4b91-806b-53db660a0b13 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:11:17.421Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailV2VirtualNetwork" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "97d8ab0d-e495-4f5b-8d16-f646685b3f9d" INVARIANT_UUID = "c9dc8a15-620c-447a-9cc1-877bc3ac9276" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "34b6e09a-b44a-45c3-a3cc-219c8bcfc9d4.assettoscacsar" CURR_ARTIFACT_UUID = "34b6e09a-b44a-45c3-a3cc-219c8bcfc9d4.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Contrailv2virtualnetwork-csar.csar,0,34b6e09a-b44a-45c3-a3cc-219c8bcfc9d4.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5167 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f3e10c03-730e-4cae-8164-5b3ce5f826b8 TargetServiceName=null EntryTimestamp=2021-08-11T16:11:12.254Z InvocationID=7c231fbe-9636-4a51-9f7c-2c2f47f4df75 ErrorCode=500 LogTimestamp=2021-08-11T16:11:17.421603Z ResponseDesc=Created Exiting. 2021-08-11T16:11:17.422Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:11:17.442Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=8138dd9a-b369-49c0-a0d8-15352506ae40 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:11:17.441817Z InvocationID=a62207b7-6597-4d50-9bc9-86cd4cf428ab LogTimestamp=2021-08-11T16:11:17.441869Z Entering 2021-08-11T16:11:17.442Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=8138dd9a-b369-49c0-a0d8-15352506ae40 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:11:17.441817Z InvocationID=a62207b7-6597-4d50-9bc9-86cd4cf428ab LogTimestamp=2021-08-11T16:11:17.441869Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:11:17.701Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:17.442Z InvocationID=964072c2-65af-43fe-a1d1-1370f6b8e044 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:17.701Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:17.442Z InvocationID=964072c2-65af-43fe-a1d1-1370f6b8e044 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:17.701Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:17.442Z InvocationID=964072c2-65af-43fe-a1d1-1370f6b8e044 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:17.701Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:17.442Z InvocationID=964072c2-65af-43fe-a1d1-1370f6b8e044 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:17.713Z [qtp2045766957-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:17.442Z InvocationID=964072c2-65af-43fe-a1d1-1370f6b8e044 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Securityrules 2021-08-11T16:11:18.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:11:18.316Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:11:18.937Z [qtp2045766957-15] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "SecurityRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "095c90c2-7a4d-4d27-8f91-e98b6b18c661" INVARIANT_UUID = "e3462674-510d-488c-a027-350880110bfa" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.rules.nodes.heat.network.neutron.SecurityRules" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=095c90c2-7a4d-4d27-8f91-e98b6b18c661 EntryTimestamp=2021-08-11T16:11:17.442Z InvocationID=964072c2-65af-43fe-a1d1-1370f6b8e044 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:11:18.937Z [qtp2045766957-15] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "SecurityRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "095c90c2-7a4d-4d27-8f91-e98b6b18c661" INVARIANT_UUID = "e3462674-510d-488c-a027-350880110bfa" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.rules.nodes.heat.network.neutron.SecurityRules" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=095c90c2-7a4d-4d27-8f91-e98b6b18c661 TargetServiceName=null EntryTimestamp=2021-08-11T16:11:17.442Z InvocationID=964072c2-65af-43fe-a1d1-1370f6b8e044 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:11:19.118Z [qtp2045766957-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "SecurityRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "095c90c2-7a4d-4d27-8f91-e98b6b18c661" INVARIANT_UUID = "e3462674-510d-488c-a027-350880110bfa" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.rules.nodes.heat.network.neutron.SecurityRules" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=095c90c2-7a4d-4d27-8f91-e98b6b18c661 TargetServiceName=null EntryTimestamp=2021-08-11T16:11:17.442Z InvocationID=964072c2-65af-43fe-a1d1-1370f6b8e044 ErrorCode=500 lock resource with id d043147d-f65e-469b-baa5-b300af1db629 2021-08-11T16:11:19.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f8a1e7c0-ac5c-4200-b85a-8afe7cbf8d36 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:11:19.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f8a1e7c0-ac5c-4200-b85a-8afe7cbf8d36 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@8e9bdb7 2021-08-11T16:11:19.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f8a1e7c0-ac5c-4200-b85a-8afe7cbf8d36 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:11:19.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f8a1e7c0-ac5c-4200-b85a-8afe7cbf8d36 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:11:20.785Z [qtp2045766957-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "SecurityRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "095c90c2-7a4d-4d27-8f91-e98b6b18c661" INVARIANT_UUID = "e3462674-510d-488c-a027-350880110bfa" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "d043147d-f65e-469b-baa5-b300af1db629.assettoscatemplate" CURR_ARTIFACT_UUID = "d043147d-f65e-469b-baa5-b300af1db629.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Securityrules-template.yml,0,d043147d-f65e-469b-baa5-b300af1db629.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=760b9574-cfdc-4f5a-b904-a2c8dfd2fc75 TargetServiceName=null EntryTimestamp=2021-08-11T16:11:17.442Z InvocationID=964072c2-65af-43fe-a1d1-1370f6b8e044 ErrorCode=500 Starting to create Csar for component SecurityRules 2021-08-11T16:11:20.922Z [qtp2045766957-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "SecurityRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "095c90c2-7a4d-4d27-8f91-e98b6b18c661" INVARIANT_UUID = "e3462674-510d-488c-a027-350880110bfa" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "d043147d-f65e-469b-baa5-b300af1db629.assettoscatemplate" CURR_ARTIFACT_UUID = "d043147d-f65e-469b-baa5-b300af1db629.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Securityrules-template.yml,0,d043147d-f65e-469b-baa5-b300af1db629.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=760b9574-cfdc-4f5a-b904-a2c8dfd2fc75 TargetServiceName=null EntryTimestamp=2021-08-11T16:11:17.442Z InvocationID=964072c2-65af-43fe-a1d1-1370f6b8e044 ErrorCode=500 Ended create Csar for component SecurityRules 2021-08-11T16:11:20.922Z [qtp2045766957-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "SecurityRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "095c90c2-7a4d-4d27-8f91-e98b6b18c661" INVARIANT_UUID = "e3462674-510d-488c-a027-350880110bfa" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "d043147d-f65e-469b-baa5-b300af1db629.assettoscatemplate" CURR_ARTIFACT_UUID = "d043147d-f65e-469b-baa5-b300af1db629.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Securityrules-template.yml,0,d043147d-f65e-469b-baa5-b300af1db629.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=760b9574-cfdc-4f5a-b904-a2c8dfd2fc75 TargetServiceName=null EntryTimestamp=2021-08-11T16:11:17.442Z InvocationID=964072c2-65af-43fe-a1d1-1370f6b8e044 ErrorCode=500 lock resource with id d043147d-f65e-469b-baa5-b300af1db629 2021-08-11T16:11:22.588Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "SecurityRules" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "095c90c2-7a4d-4d27-8f91-e98b6b18c661" INVARIANT_UUID = "e3462674-510d-488c-a027-350880110bfa" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "d043147d-f65e-469b-baa5-b300af1db629.assettoscacsar" CURR_ARTIFACT_UUID = "d043147d-f65e-469b-baa5-b300af1db629.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Securityrules-csar.csar,0,d043147d-f65e-469b-baa5-b300af1db629.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5145 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=6d16498e-afac-4328-9514-b4e4f580e83e TargetServiceName=null EntryTimestamp=2021-08-11T16:11:17.442Z InvocationID=964072c2-65af-43fe-a1d1-1370f6b8e044 ErrorCode=500 LogTimestamp=2021-08-11T16:11:22.587688Z ResponseDesc=Created Exiting. 2021-08-11T16:11:22.589Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:11:22.607Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=520be9b0-c80d-451a-a760-c2b98517aa3c InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:11:22.607547Z InvocationID=c876156c-42d4-4cfb-9d66-d4be308c9f22 LogTimestamp=2021-08-11T16:11:22.607589Z Entering 2021-08-11T16:11:22.608Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=520be9b0-c80d-451a-a760-c2b98517aa3c InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:11:22.607547Z InvocationID=c876156c-42d4-4cfb-9d66-d4be308c9f22 LogTimestamp=2021-08-11T16:11:22.607589Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:11:22.693Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee5cde6b-f436-4a9a-90dd-f6d4d5423a3b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:11:22.693Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee5cde6b-f436-4a9a-90dd-f6d4d5423a3b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@44ce5a83 2021-08-11T16:11:22.693Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee5cde6b-f436-4a9a-90dd-f6d4d5423a3b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:11:22.693Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee5cde6b-f436-4a9a-90dd-f6d4d5423a3b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:11:23.016Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:22.608Z InvocationID=bfa61fea-136c-44d4-b262-21bef6a1a0f0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:23.016Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:22.608Z InvocationID=bfa61fea-136c-44d4-b262-21bef6a1a0f0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:23.016Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:22.608Z InvocationID=bfa61fea-136c-44d4-b262-21bef6a1a0f0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:23.016Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:22.608Z InvocationID=bfa61fea-136c-44d4-b262-21bef6a1a0f0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:23.016Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:22.608Z InvocationID=bfa61fea-136c-44d4-b262-21bef6a1a0f0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Contrailabstractsubstitute 2021-08-11T16:11:24.479Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailAbstractSubstitute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "c51d9dfb-ec86-4f79-84dc-08917f5a2dde" INVARIANT_UUID = "c2de30c2-c707-476c-bd44-e6eb96bc2695" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.abstract.nodes.contrail.AbstractSubstitute" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c51d9dfb-ec86-4f79-84dc-08917f5a2dde EntryTimestamp=2021-08-11T16:11:22.608Z InvocationID=bfa61fea-136c-44d4-b262-21bef6a1a0f0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id d5e43029-6c7b-46ee-98dc-2ca6405a1bdc 2021-08-11T16:11:25.749Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5a61091-cc36-45f3-a994-d5c443837d8d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:11:25.749Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5a61091-cc36-45f3-a994-d5c443837d8d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@bde2a1c 2021-08-11T16:11:25.750Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5a61091-cc36-45f3-a994-d5c443837d8d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:11:25.750Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5a61091-cc36-45f3-a994-d5c443837d8d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:11:26.683Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailAbstractSubstitute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "c51d9dfb-ec86-4f79-84dc-08917f5a2dde" INVARIANT_UUID = "c2de30c2-c707-476c-bd44-e6eb96bc2695" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "d5e43029-6c7b-46ee-98dc-2ca6405a1bdc.assettoscatemplate" CURR_ARTIFACT_UUID = "d5e43029-6c7b-46ee-98dc-2ca6405a1bdc.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailabstractsubstitute-template.yml,0,d5e43029-6c7b-46ee-98dc-2ca6405a1bdc.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=4073 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ffc8199d-7e06-457b-8366-781dc5e2709b EntryTimestamp=2021-08-11T16:11:22.608Z InvocationID=bfa61fea-136c-44d4-b262-21bef6a1a0f0 LogTimestamp=2021-08-11T16:11:26.681831Z ResponseDesc=Created Exiting. 2021-08-11T16:11:26.684Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:11:26.755Z [qtp2045766957-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=120fd639-51c9-41c1-b61a-249d14da529c InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:11:26.754925Z InvocationID=31b40816-8977-440b-ba67-f37f2f3f679e LogTimestamp=2021-08-11T16:11:26.755023Z Entering 2021-08-11T16:11:26.755Z [qtp2045766957-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=120fd639-51c9-41c1-b61a-249d14da529c InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:11:26.754925Z InvocationID=31b40816-8977-440b-ba67-f37f2f3f679e LogTimestamp=2021-08-11T16:11:26.755023Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:11:27.768Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:26.755Z InvocationID=7f9885c1-52e6-4769-aae5-dc09d73910e9 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:27.768Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:26.755Z InvocationID=7f9885c1-52e6-4769-aae5-dc09d73910e9 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:27.768Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:26.755Z InvocationID=7f9885c1-52e6-4769-aae5-dc09d73910e9 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:27.768Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:26.755Z InvocationID=7f9885c1-52e6-4769-aae5-dc09d73910e9 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:27.768Z [qtp2045766957-11] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:26.755Z InvocationID=7f9885c1-52e6-4769-aae5-dc09d73910e9 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Contrailcompute 2021-08-11T16:11:28.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a69f18ad-0a05-4376-a337-dddc53a9ad84 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:11:28.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a69f18ad-0a05-4376-a337-dddc53a9ad84 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@48dca079 2021-08-11T16:11:28.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a69f18ad-0a05-4376-a337-dddc53a9ad84 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:11:28.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a69f18ad-0a05-4376-a337-dddc53a9ad84 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:11:29.662Z [qtp2045766957-11] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailCompute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "9b60e656-8af7-40a4-bd85-529f7ec42c1c" INVARIANT_UUID = "d3308a33-3e73-4eb2-95c5-a542fa5bfa64" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.nodes.heat.contrail.Compute" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9b60e656-8af7-40a4-bd85-529f7ec42c1c EntryTimestamp=2021-08-11T16:11:26.755Z InvocationID=7f9885c1-52e6-4769-aae5-dc09d73910e9 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:11:29.663Z [qtp2045766957-11] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailCompute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "9b60e656-8af7-40a4-bd85-529f7ec42c1c" INVARIANT_UUID = "d3308a33-3e73-4eb2-95c5-a542fa5bfa64" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.nodes.heat.contrail.Compute" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9b60e656-8af7-40a4-bd85-529f7ec42c1c TargetServiceName=null EntryTimestamp=2021-08-11T16:11:26.755Z InvocationID=7f9885c1-52e6-4769-aae5-dc09d73910e9 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:11:30.301Z [qtp2045766957-11] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailCompute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "9b60e656-8af7-40a4-bd85-529f7ec42c1c" INVARIANT_UUID = "d3308a33-3e73-4eb2-95c5-a542fa5bfa64" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.nodes.heat.contrail.Compute" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9b60e656-8af7-40a4-bd85-529f7ec42c1c TargetServiceName=null EntryTimestamp=2021-08-11T16:11:26.755Z InvocationID=7f9885c1-52e6-4769-aae5-dc09d73910e9 ErrorCode=500 lock resource with id d5fb3cf2-dd61-4ce0-a1d3-c1b19bb148a0 2021-08-11T16:11:31.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c19f4bc3-348a-424d-b2de-91e736d1719b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:11:31.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c19f4bc3-348a-424d-b2de-91e736d1719b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@d1da8d0 2021-08-11T16:11:31.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c19f4bc3-348a-424d-b2de-91e736d1719b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:11:31.654Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c19f4bc3-348a-424d-b2de-91e736d1719b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:11:31.903Z [qtp2045766957-11] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailCompute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "9b60e656-8af7-40a4-bd85-529f7ec42c1c" INVARIANT_UUID = "d3308a33-3e73-4eb2-95c5-a542fa5bfa64" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "d5fb3cf2-dd61-4ce0-a1d3-c1b19bb148a0.assettoscatemplate" CURR_ARTIFACT_UUID = "d5fb3cf2-dd61-4ce0-a1d3-c1b19bb148a0.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailcompute-template.yml,0,d5fb3cf2-dd61-4ce0-a1d3-c1b19bb148a0.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2cc5e20f-2b4c-47d0-a621-04639ce74646 TargetServiceName=null EntryTimestamp=2021-08-11T16:11:26.755Z InvocationID=7f9885c1-52e6-4769-aae5-dc09d73910e9 ErrorCode=500 Starting to create Csar for component ContrailCompute 2021-08-11T16:11:32.112Z [qtp2045766957-11] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailCompute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "9b60e656-8af7-40a4-bd85-529f7ec42c1c" INVARIANT_UUID = "d3308a33-3e73-4eb2-95c5-a542fa5bfa64" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "d5fb3cf2-dd61-4ce0-a1d3-c1b19bb148a0.assettoscatemplate" CURR_ARTIFACT_UUID = "d5fb3cf2-dd61-4ce0-a1d3-c1b19bb148a0.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailcompute-template.yml,0,d5fb3cf2-dd61-4ce0-a1d3-c1b19bb148a0.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2cc5e20f-2b4c-47d0-a621-04639ce74646 TargetServiceName=null EntryTimestamp=2021-08-11T16:11:26.755Z InvocationID=7f9885c1-52e6-4769-aae5-dc09d73910e9 ErrorCode=500 Ended create Csar for component ContrailCompute 2021-08-11T16:11:32.113Z [qtp2045766957-11] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailCompute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "9b60e656-8af7-40a4-bd85-529f7ec42c1c" INVARIANT_UUID = "d3308a33-3e73-4eb2-95c5-a542fa5bfa64" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "d5fb3cf2-dd61-4ce0-a1d3-c1b19bb148a0.assettoscatemplate" CURR_ARTIFACT_UUID = "d5fb3cf2-dd61-4ce0-a1d3-c1b19bb148a0.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailcompute-template.yml,0,d5fb3cf2-dd61-4ce0-a1d3-c1b19bb148a0.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2cc5e20f-2b4c-47d0-a621-04639ce74646 TargetServiceName=null EntryTimestamp=2021-08-11T16:11:26.755Z InvocationID=7f9885c1-52e6-4769-aae5-dc09d73910e9 ErrorCode=500 lock resource with id d5fb3cf2-dd61-4ce0-a1d3-c1b19bb148a0 2021-08-11T16:11:33.309Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:11:33.316Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:11:33.739Z [qtp2045766957-11] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailCompute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "9b60e656-8af7-40a4-bd85-529f7ec42c1c" INVARIANT_UUID = "d3308a33-3e73-4eb2-95c5-a542fa5bfa64" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "d5fb3cf2-dd61-4ce0-a1d3-c1b19bb148a0.assettoscacsar" CURR_ARTIFACT_UUID = "d5fb3cf2-dd61-4ce0-a1d3-c1b19bb148a0.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Contrailcompute-csar.csar,0,d5fb3cf2-dd61-4ce0-a1d3-c1b19bb148a0.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=6984 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=351d4a69-c710-4a7c-8fed-721502dc3147 TargetServiceName=null EntryTimestamp=2021-08-11T16:11:26.755Z InvocationID=7f9885c1-52e6-4769-aae5-dc09d73910e9 ErrorCode=500 LogTimestamp=2021-08-11T16:11:33.739009Z ResponseDesc=Created Exiting. 2021-08-11T16:11:33.740Z [qtp2045766957-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:11:33.764Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=dc5b0a05-1ecd-423e-8b7f-6640df1fa84c InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:11:33.764010Z InvocationID=9d88543a-f274-4319-89ef-eeaec9c36450 LogTimestamp=2021-08-11T16:11:33.764056Z Entering 2021-08-11T16:11:33.764Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=dc5b0a05-1ecd-423e-8b7f-6640df1fa84c InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:11:33.764010Z InvocationID=9d88543a-f274-4319-89ef-eeaec9c36450 LogTimestamp=2021-08-11T16:11:33.764056Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:11:34.065Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:33.764Z InvocationID=bb6fae71-e893-42e9-a516-ef5c997d1076 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:34.066Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:33.764Z InvocationID=bb6fae71-e893-42e9-a516-ef5c997d1076 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:34.066Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:33.764Z InvocationID=bb6fae71-e893-42e9-a516-ef5c997d1076 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:34.066Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:33.764Z InvocationID=bb6fae71-e893-42e9-a516-ef5c997d1076 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:34.080Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:33.764Z InvocationID=bb6fae71-e893-42e9-a516-ef5c997d1076 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Contrailv2virtualmachineinterface 2021-08-11T16:11:34.703Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55abcfe4-66a5-464f-b586-db2d8a133e22 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:11:34.703Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55abcfe4-66a5-464f-b586-db2d8a133e22 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3b31e1b2 2021-08-11T16:11:34.703Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55abcfe4-66a5-464f-b586-db2d8a133e22 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:11:34.703Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55abcfe4-66a5-464f-b586-db2d8a133e22 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:11:35.575Z [qtp2045766957-12] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailV2VirtualMachineInterface" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "50f42ad5-c7ee-47c3-b4b2-2c2635b675b0" INVARIANT_UUID = "ec9d7e04-c2b1-4171-a532-cdfb22cff507" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.heat.contrailV2.VirtualMachineInterface" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=50f42ad5-c7ee-47c3-b4b2-2c2635b675b0 EntryTimestamp=2021-08-11T16:11:33.764Z InvocationID=bb6fae71-e893-42e9-a516-ef5c997d1076 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:11:35.576Z [qtp2045766957-12] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailV2VirtualMachineInterface" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "50f42ad5-c7ee-47c3-b4b2-2c2635b675b0" INVARIANT_UUID = "ec9d7e04-c2b1-4171-a532-cdfb22cff507" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.heat.contrailV2.VirtualMachineInterface" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=50f42ad5-c7ee-47c3-b4b2-2c2635b675b0 TargetServiceName=null EntryTimestamp=2021-08-11T16:11:33.764Z InvocationID=bb6fae71-e893-42e9-a516-ef5c997d1076 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:11:35.790Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ContrailV2VirtualMachineInterface" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "50f42ad5-c7ee-47c3-b4b2-2c2635b675b0" INVARIANT_UUID = "ec9d7e04-c2b1-4171-a532-cdfb22cff507" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.heat.contrailV2.VirtualMachineInterface" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=50f42ad5-c7ee-47c3-b4b2-2c2635b675b0 TargetServiceName=null EntryTimestamp=2021-08-11T16:11:33.764Z InvocationID=bb6fae71-e893-42e9-a516-ef5c997d1076 ErrorCode=500 lock resource with id b1c81de8-01ce-4ce8-ba6f-ef2cca15f5a6 2021-08-11T16:11:37.398Z [qtp2045766957-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailV2VirtualMachineInterface" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "50f42ad5-c7ee-47c3-b4b2-2c2635b675b0" INVARIANT_UUID = "ec9d7e04-c2b1-4171-a532-cdfb22cff507" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b1c81de8-01ce-4ce8-ba6f-ef2cca15f5a6.assettoscatemplate" CURR_ARTIFACT_UUID = "b1c81de8-01ce-4ce8-ba6f-ef2cca15f5a6.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailv2virtualmachineinterface-template.yml,0,b1c81de8-01ce-4ce8-ba6f-ef2cca15f5a6.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=03e0027c-761e-4053-932a-0f4b5421d005 TargetServiceName=null EntryTimestamp=2021-08-11T16:11:33.764Z InvocationID=bb6fae71-e893-42e9-a516-ef5c997d1076 ErrorCode=500 Starting to create Csar for component ContrailV2VirtualMachineInterface 2021-08-11T16:11:37.546Z [qtp2045766957-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailV2VirtualMachineInterface" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "50f42ad5-c7ee-47c3-b4b2-2c2635b675b0" INVARIANT_UUID = "ec9d7e04-c2b1-4171-a532-cdfb22cff507" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b1c81de8-01ce-4ce8-ba6f-ef2cca15f5a6.assettoscatemplate" CURR_ARTIFACT_UUID = "b1c81de8-01ce-4ce8-ba6f-ef2cca15f5a6.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailv2virtualmachineinterface-template.yml,0,b1c81de8-01ce-4ce8-ba6f-ef2cca15f5a6.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=03e0027c-761e-4053-932a-0f4b5421d005 TargetServiceName=null EntryTimestamp=2021-08-11T16:11:33.764Z InvocationID=bb6fae71-e893-42e9-a516-ef5c997d1076 ErrorCode=500 Ended create Csar for component ContrailV2VirtualMachineInterface 2021-08-11T16:11:37.546Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailV2VirtualMachineInterface" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "50f42ad5-c7ee-47c3-b4b2-2c2635b675b0" INVARIANT_UUID = "ec9d7e04-c2b1-4171-a532-cdfb22cff507" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b1c81de8-01ce-4ce8-ba6f-ef2cca15f5a6.assettoscatemplate" CURR_ARTIFACT_UUID = "b1c81de8-01ce-4ce8-ba6f-ef2cca15f5a6.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailv2virtualmachineinterface-template.yml,0,b1c81de8-01ce-4ce8-ba6f-ef2cca15f5a6.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=03e0027c-761e-4053-932a-0f4b5421d005 TargetServiceName=null EntryTimestamp=2021-08-11T16:11:33.764Z InvocationID=bb6fae71-e893-42e9-a516-ef5c997d1076 ErrorCode=500 lock resource with id b1c81de8-01ce-4ce8-ba6f-ef2cca15f5a6 2021-08-11T16:11:37.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=88c15b63-fa52-483f-bb84-49303c586fac PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:11:37.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=88c15b63-fa52-483f-bb84-49303c586fac PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@41c5b202 2021-08-11T16:11:37.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=88c15b63-fa52-483f-bb84-49303c586fac PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:11:37.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=88c15b63-fa52-483f-bb84-49303c586fac PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:11:39.391Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ContrailV2VirtualMachineInterface" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "50f42ad5-c7ee-47c3-b4b2-2c2635b675b0" INVARIANT_UUID = "ec9d7e04-c2b1-4171-a532-cdfb22cff507" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b1c81de8-01ce-4ce8-ba6f-ef2cca15f5a6.assettoscacsar" CURR_ARTIFACT_UUID = "b1c81de8-01ce-4ce8-ba6f-ef2cca15f5a6.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Contrailv2virtualmachineinterface-csar.csar,0,b1c81de8-01ce-4ce8-ba6f-ef2cca15f5a6.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5626 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9cdbfa78-7b3e-48d8-a261-17ca44958585 TargetServiceName=null EntryTimestamp=2021-08-11T16:11:33.764Z InvocationID=bb6fae71-e893-42e9-a516-ef5c997d1076 ErrorCode=500 LogTimestamp=2021-08-11T16:11:39.390919Z ResponseDesc=Created Exiting. 2021-08-11T16:11:39.391Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:11:39.454Z [qtp2045766957-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=12b96696-273e-410e-957f-7ca207721e1d InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:11:39.454099Z InvocationID=fc20f298-7446-43bd-a2a5-27e31dd3e3ab LogTimestamp=2021-08-11T16:11:39.454151Z Entering 2021-08-11T16:11:39.454Z [qtp2045766957-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=12b96696-273e-410e-957f-7ca207721e1d InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:11:39.454099Z InvocationID=fc20f298-7446-43bd-a2a5-27e31dd3e3ab LogTimestamp=2021-08-11T16:11:39.454151Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:11:40.265Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:39.454Z InvocationID=84065de2-baeb-461e-a31c-fb594fe959fb PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:40.265Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:39.454Z InvocationID=84065de2-baeb-461e-a31c-fb594fe959fb PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:40.265Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:39.454Z InvocationID=84065de2-baeb-461e-a31c-fb594fe959fb PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:40.265Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:39.454Z InvocationID=84065de2-baeb-461e-a31c-fb594fe959fb PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:40.265Z [qtp2045766957-11] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:39.454Z InvocationID=84065de2-baeb-461e-a31c-fb594fe959fb PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Subinterface 2021-08-11T16:11:40.849Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a7dc51a-a861-4774-9d5d-524a641b252b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:11:40.849Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a7dc51a-a861-4774-9d5d-524a641b252b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6cb71d2b 2021-08-11T16:11:40.849Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a7dc51a-a861-4774-9d5d-524a641b252b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:11:40.849Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a7dc51a-a861-4774-9d5d-524a641b252b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:11:41.966Z [qtp2045766957-11] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "subInterface" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "07f78884-3b8b-4b31-95e2-7c6447f373dc" INVARIANT_UUID = "9b5785f8-60f2-4e7b-833a-df6478eed39e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.network.SubInterface" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=07f78884-3b8b-4b31-95e2-7c6447f373dc EntryTimestamp=2021-08-11T16:11:39.454Z InvocationID=84065de2-baeb-461e-a31c-fb594fe959fb PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:11:41.967Z [qtp2045766957-11] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "subInterface" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "07f78884-3b8b-4b31-95e2-7c6447f373dc" INVARIANT_UUID = "9b5785f8-60f2-4e7b-833a-df6478eed39e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.network.SubInterface" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=07f78884-3b8b-4b31-95e2-7c6447f373dc TargetServiceName=null EntryTimestamp=2021-08-11T16:11:39.454Z InvocationID=84065de2-baeb-461e-a31c-fb594fe959fb ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:11:42.245Z [qtp2045766957-11] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "subInterface" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "07f78884-3b8b-4b31-95e2-7c6447f373dc" INVARIANT_UUID = "9b5785f8-60f2-4e7b-833a-df6478eed39e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.network.SubInterface" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=07f78884-3b8b-4b31-95e2-7c6447f373dc TargetServiceName=null EntryTimestamp=2021-08-11T16:11:39.454Z InvocationID=84065de2-baeb-461e-a31c-fb594fe959fb ErrorCode=500 lock resource with id 05aab253-df78-48e4-98d2-62a23de9e232 2021-08-11T16:11:43.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8aa1c05a-c119-4a16-9fa4-02685200c2ba PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:11:43.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8aa1c05a-c119-4a16-9fa4-02685200c2ba PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@c8c6345 2021-08-11T16:11:43.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8aa1c05a-c119-4a16-9fa4-02685200c2ba PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:11:43.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8aa1c05a-c119-4a16-9fa4-02685200c2ba PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:11:43.861Z [qtp2045766957-11] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "subInterface" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "07f78884-3b8b-4b31-95e2-7c6447f373dc" INVARIANT_UUID = "9b5785f8-60f2-4e7b-833a-df6478eed39e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "05aab253-df78-48e4-98d2-62a23de9e232.assettoscatemplate" CURR_ARTIFACT_UUID = "05aab253-df78-48e4-98d2-62a23de9e232.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Subinterface-template.yml,0,05aab253-df78-48e4-98d2-62a23de9e232.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=0d583522-91bc-42db-90d0-ea024d3b692b TargetServiceName=null EntryTimestamp=2021-08-11T16:11:39.454Z InvocationID=84065de2-baeb-461e-a31c-fb594fe959fb ErrorCode=500 Starting to create Csar for component subInterface 2021-08-11T16:11:44.034Z [qtp2045766957-11] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "subInterface" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "07f78884-3b8b-4b31-95e2-7c6447f373dc" INVARIANT_UUID = "9b5785f8-60f2-4e7b-833a-df6478eed39e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "05aab253-df78-48e4-98d2-62a23de9e232.assettoscatemplate" CURR_ARTIFACT_UUID = "05aab253-df78-48e4-98d2-62a23de9e232.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Subinterface-template.yml,0,05aab253-df78-48e4-98d2-62a23de9e232.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=0d583522-91bc-42db-90d0-ea024d3b692b TargetServiceName=null EntryTimestamp=2021-08-11T16:11:39.454Z InvocationID=84065de2-baeb-461e-a31c-fb594fe959fb ErrorCode=500 Ended create Csar for component subInterface 2021-08-11T16:11:44.035Z [qtp2045766957-11] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "subInterface" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "07f78884-3b8b-4b31-95e2-7c6447f373dc" INVARIANT_UUID = "9b5785f8-60f2-4e7b-833a-df6478eed39e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "05aab253-df78-48e4-98d2-62a23de9e232.assettoscatemplate" CURR_ARTIFACT_UUID = "05aab253-df78-48e4-98d2-62a23de9e232.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Subinterface-template.yml,0,05aab253-df78-48e4-98d2-62a23de9e232.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=0d583522-91bc-42db-90d0-ea024d3b692b TargetServiceName=null EntryTimestamp=2021-08-11T16:11:39.454Z InvocationID=84065de2-baeb-461e-a31c-fb594fe959fb ErrorCode=500 lock resource with id 05aab253-df78-48e4-98d2-62a23de9e232 2021-08-11T16:11:45.664Z [qtp2045766957-11] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "subInterface" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "07f78884-3b8b-4b31-95e2-7c6447f373dc" INVARIANT_UUID = "9b5785f8-60f2-4e7b-833a-df6478eed39e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "05aab253-df78-48e4-98d2-62a23de9e232.assettoscacsar" CURR_ARTIFACT_UUID = "05aab253-df78-48e4-98d2-62a23de9e232.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Subinterface-csar.csar,0,05aab253-df78-48e4-98d2-62a23de9e232.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=6210 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bd18b043-55ca-4e8c-9276-ff5c5feefb5e TargetServiceName=null EntryTimestamp=2021-08-11T16:11:39.454Z InvocationID=84065de2-baeb-461e-a31c-fb594fe959fb ErrorCode=500 LogTimestamp=2021-08-11T16:11:45.664272Z ResponseDesc=Created Exiting. 2021-08-11T16:11:45.665Z [qtp2045766957-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:11:45.685Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=efbec453-defc-4e75-9678-5fbfcee2c198 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:11:45.685100Z InvocationID=221c4d7e-0ff3-4d2b-a215-07f2480473c1 LogTimestamp=2021-08-11T16:11:45.685149Z Entering 2021-08-11T16:11:45.685Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=efbec453-defc-4e75-9678-5fbfcee2c198 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:11:45.685100Z InvocationID=221c4d7e-0ff3-4d2b-a215-07f2480473c1 LogTimestamp=2021-08-11T16:11:45.685149Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:11:46.031Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:45.685Z InvocationID=7216ad67-2027-467d-bcfa-1b7597b639fe PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:46.031Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:45.685Z InvocationID=7216ad67-2027-467d-bcfa-1b7597b639fe PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:46.031Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:45.685Z InvocationID=7216ad67-2027-467d-bcfa-1b7597b639fe PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:46.031Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:45.685Z InvocationID=7216ad67-2027-467d-bcfa-1b7597b639fe PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:46.042Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:45.685Z InvocationID=7216ad67-2027-467d-bcfa-1b7597b639fe PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Contrailv2vlansubinterface 2021-08-11T16:11:46.687Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ffba1695-4783-4660-85c8-f076766846ff PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:11:46.688Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ffba1695-4783-4660-85c8-f076766846ff PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@38842f17 2021-08-11T16:11:46.688Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ffba1695-4783-4660-85c8-f076766846ff PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:11:46.688Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ffba1695-4783-4660-85c8-f076766846ff PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:11:47.396Z [qtp2045766957-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "contrailV2VLANSubInterface" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "541ecf6c-0a92-4dba-a14c-cc094c29f42e" INVARIANT_UUID = "3532f719-c371-4726-aca4-3d4ff3288f3e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.heat.network.contrailV2.VLANSubInterface" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=541ecf6c-0a92-4dba-a14c-cc094c29f42e EntryTimestamp=2021-08-11T16:11:45.685Z InvocationID=7216ad67-2027-467d-bcfa-1b7597b639fe PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:11:47.396Z [qtp2045766957-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "contrailV2VLANSubInterface" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "541ecf6c-0a92-4dba-a14c-cc094c29f42e" INVARIANT_UUID = "3532f719-c371-4726-aca4-3d4ff3288f3e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.heat.network.contrailV2.VLANSubInterface" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=541ecf6c-0a92-4dba-a14c-cc094c29f42e TargetServiceName=null EntryTimestamp=2021-08-11T16:11:45.685Z InvocationID=7216ad67-2027-467d-bcfa-1b7597b639fe ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:11:47.597Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "contrailV2VLANSubInterface" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "541ecf6c-0a92-4dba-a14c-cc094c29f42e" INVARIANT_UUID = "3532f719-c371-4726-aca4-3d4ff3288f3e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.heat.network.contrailV2.VLANSubInterface" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=541ecf6c-0a92-4dba-a14c-cc094c29f42e TargetServiceName=null EntryTimestamp=2021-08-11T16:11:45.685Z InvocationID=7216ad67-2027-467d-bcfa-1b7597b639fe ErrorCode=500 lock resource with id b892b636-548a-4532-a707-84c25bd21d99 2021-08-11T16:11:48.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:11:48.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:11:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:11:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:11:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:11:49.355Z [qtp2045766957-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "contrailV2VLANSubInterface" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "541ecf6c-0a92-4dba-a14c-cc094c29f42e" INVARIANT_UUID = "3532f719-c371-4726-aca4-3d4ff3288f3e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b892b636-548a-4532-a707-84c25bd21d99.assettoscatemplate" CURR_ARTIFACT_UUID = "b892b636-548a-4532-a707-84c25bd21d99.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailv2vlansubinterface-template.yml,0,b892b636-548a-4532-a707-84c25bd21d99.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=4a87c965-251b-41c0-adbe-ffe130aa05d9 TargetServiceName=null EntryTimestamp=2021-08-11T16:11:45.685Z InvocationID=7216ad67-2027-467d-bcfa-1b7597b639fe ErrorCode=500 Starting to create Csar for component contrailV2VLANSubInterface 2021-08-11T16:11:49.674Z [qtp2045766957-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "contrailV2VLANSubInterface" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "541ecf6c-0a92-4dba-a14c-cc094c29f42e" INVARIANT_UUID = "3532f719-c371-4726-aca4-3d4ff3288f3e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b892b636-548a-4532-a707-84c25bd21d99.assettoscatemplate" CURR_ARTIFACT_UUID = "b892b636-548a-4532-a707-84c25bd21d99.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailv2vlansubinterface-template.yml,0,b892b636-548a-4532-a707-84c25bd21d99.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=4a87c965-251b-41c0-adbe-ffe130aa05d9 TargetServiceName=null EntryTimestamp=2021-08-11T16:11:45.685Z InvocationID=7216ad67-2027-467d-bcfa-1b7597b639fe ErrorCode=500 Ended create Csar for component contrailV2VLANSubInterface 2021-08-11T16:11:49.674Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "contrailV2VLANSubInterface" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "541ecf6c-0a92-4dba-a14c-cc094c29f42e" INVARIANT_UUID = "3532f719-c371-4726-aca4-3d4ff3288f3e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b892b636-548a-4532-a707-84c25bd21d99.assettoscatemplate" CURR_ARTIFACT_UUID = "b892b636-548a-4532-a707-84c25bd21d99.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailv2vlansubinterface-template.yml,0,b892b636-548a-4532-a707-84c25bd21d99.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=4a87c965-251b-41c0-adbe-ffe130aa05d9 TargetServiceName=null EntryTimestamp=2021-08-11T16:11:45.685Z InvocationID=7216ad67-2027-467d-bcfa-1b7597b639fe ErrorCode=500 lock resource with id b892b636-548a-4532-a707-84c25bd21d99 2021-08-11T16:11:50.160Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d228ba02-ea5f-4205-9b14-91499cbed297 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:11:50.161Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d228ba02-ea5f-4205-9b14-91499cbed297 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@543ddfa7 2021-08-11T16:11:50.161Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d228ba02-ea5f-4205-9b14-91499cbed297 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:11:50.161Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d228ba02-ea5f-4205-9b14-91499cbed297 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:11:51.563Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "contrailV2VLANSubInterface" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "541ecf6c-0a92-4dba-a14c-cc094c29f42e" INVARIANT_UUID = "3532f719-c371-4726-aca4-3d4ff3288f3e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b892b636-548a-4532-a707-84c25bd21d99.assettoscacsar" CURR_ARTIFACT_UUID = "b892b636-548a-4532-a707-84c25bd21d99.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Contrailv2vlansubinterface-csar.csar,0,b892b636-548a-4532-a707-84c25bd21d99.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5877 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=71b41d4e-ab23-4699-8f8a-ca38c7bd97e3 TargetServiceName=null EntryTimestamp=2021-08-11T16:11:45.685Z InvocationID=7216ad67-2027-467d-bcfa-1b7597b639fe ErrorCode=500 LogTimestamp=2021-08-11T16:11:51.562798Z ResponseDesc=Created Exiting. 2021-08-11T16:11:51.565Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:11:51.585Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=2c2662b3-c7b1-445a-91ce-bfccbcb8f500 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:11:51.585225Z InvocationID=69520faf-2042-4769-b6b3-598c57096c05 LogTimestamp=2021-08-11T16:11:51.585283Z Entering 2021-08-11T16:11:51.585Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=2c2662b3-c7b1-445a-91ce-bfccbcb8f500 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:11:51.585225Z InvocationID=69520faf-2042-4769-b6b3-598c57096c05 LogTimestamp=2021-08-11T16:11:51.585283Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:11:53.465Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21150b5c-08be-49e7-ac0e-c34edca52cb4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:11:53.465Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21150b5c-08be-49e7-ac0e-c34edca52cb4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@b7de4e 2021-08-11T16:11:53.465Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21150b5c-08be-49e7-ac0e-c34edca52cb4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:11:53.465Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21150b5c-08be-49e7-ac0e-c34edca52cb4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:11:53.468Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:51.586Z InvocationID=869a72c0-aa9c-4d6e-bbba-69aaff02360a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:53.469Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:51.586Z InvocationID=869a72c0-aa9c-4d6e-bbba-69aaff02360a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:53.469Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:51.586Z InvocationID=869a72c0-aa9c-4d6e-bbba-69aaff02360a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:53.469Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:51.586Z InvocationID=869a72c0-aa9c-4d6e-bbba-69aaff02360a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:53.469Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:51.586Z InvocationID=869a72c0-aa9c-4d6e-bbba-69aaff02360a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Multiflavorvfc 2021-08-11T16:11:56.261Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ff9a0ebd-797b-4366-a09e-5b2d9a07277e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:11:56.261Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ff9a0ebd-797b-4366-a09e-5b2d9a07277e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6f4e0356 2021-08-11T16:11:56.262Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ff9a0ebd-797b-4366-a09e-5b2d9a07277e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:11:56.262Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ff9a0ebd-797b-4366-a09e-5b2d9a07277e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:11:56.266Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "multiFlavorVFC" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "3a462807-e798-4aaf-a443-5160092ee629" INVARIANT_UUID = "bddf8ab0-5726-44ee-b82d-f381f855176a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.abstract.nodes.MultiFlavorVFC" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3a462807-e798-4aaf-a443-5160092ee629 EntryTimestamp=2021-08-11T16:11:51.586Z InvocationID=869a72c0-aa9c-4d6e-bbba-69aaff02360a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id c26d9e1b-a6a8-4dea-96f1-0a87962bb0aa 2021-08-11T16:11:58.295Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "multiFlavorVFC" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "3a462807-e798-4aaf-a443-5160092ee629" INVARIANT_UUID = "bddf8ab0-5726-44ee-b82d-f381f855176a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c26d9e1b-a6a8-4dea-96f1-0a87962bb0aa.assettoscatemplate" CURR_ARTIFACT_UUID = "c26d9e1b-a6a8-4dea-96f1-0a87962bb0aa.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Multiflavorvfc-template.yml,0,c26d9e1b-a6a8-4dea-96f1-0a87962bb0aa.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=6709 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c1863283-dc8c-4ffe-a917-27712a15d7ca EntryTimestamp=2021-08-11T16:11:51.586Z InvocationID=869a72c0-aa9c-4d6e-bbba-69aaff02360a LogTimestamp=2021-08-11T16:11:58.295001Z ResponseDesc=Created Exiting. 2021-08-11T16:11:58.295Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:11:58.368Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=01ecec39-efab-4229-9b70-f4367d0a0275 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:11:58.368255Z InvocationID=1b843945-c657-404c-b1ce-c52e80ab0ef2 LogTimestamp=2021-08-11T16:11:58.368317Z Entering 2021-08-11T16:11:58.369Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=01ecec39-efab-4229-9b70-f4367d0a0275 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:11:58.368255Z InvocationID=1b843945-c657-404c-b1ce-c52e80ab0ef2 LogTimestamp=2021-08-11T16:11:58.368317Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:11:58.712Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7dcfd47-79a4-4262-a1c7-519ca4ec3473 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:11:58.712Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7dcfd47-79a4-4262-a1c7-519ca4ec3473 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@76ab653f 2021-08-11T16:11:58.713Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7dcfd47-79a4-4262-a1c7-519ca4ec3473 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:11:58.713Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7dcfd47-79a4-4262-a1c7-519ca4ec3473 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:11:58.960Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:58.369Z InvocationID=e25c164c-9f8b-4347-ae06-de410a7ddeb2 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:58.960Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:58.369Z InvocationID=e25c164c-9f8b-4347-ae06-de410a7ddeb2 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:58.960Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:58.369Z InvocationID=e25c164c-9f8b-4347-ae06-de410a7ddeb2 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:58.960Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:58.369Z InvocationID=e25c164c-9f8b-4347-ae06-de410a7ddeb2 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:11:58.960Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:11:58.369Z InvocationID=e25c164c-9f8b-4347-ae06-de410a7ddeb2 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Vnfconfiguration 2021-08-11T16:12:00.444Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "vnfConfiguration" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "fd30aaa5-6aec-49e1-8e4e-90db29d0346e" INVARIANT_UUID = "05c54778-8135-41e3-aefc-619d767c7dd5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.abstract.nodes.VnfConfiguration" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=fd30aaa5-6aec-49e1-8e4e-90db29d0346e EntryTimestamp=2021-08-11T16:11:58.369Z InvocationID=e25c164c-9f8b-4347-ae06-de410a7ddeb2 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id 51cc5f14-99f5-410e-90f3-007281d098f8 2021-08-11T16:12:01.711Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55d09243-0daf-40b6-b7e5-51598f620f9a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:12:01.711Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55d09243-0daf-40b6-b7e5-51598f620f9a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4c514caa 2021-08-11T16:12:01.711Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55d09243-0daf-40b6-b7e5-51598f620f9a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:12:01.711Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55d09243-0daf-40b6-b7e5-51598f620f9a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:12:02.082Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "vnfConfiguration" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "fd30aaa5-6aec-49e1-8e4e-90db29d0346e" INVARIANT_UUID = "05c54778-8135-41e3-aefc-619d767c7dd5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "51cc5f14-99f5-410e-90f3-007281d098f8.assettoscatemplate" CURR_ARTIFACT_UUID = "51cc5f14-99f5-410e-90f3-007281d098f8.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vnfconfiguration-template.yml,0,51cc5f14-99f5-410e-90f3-007281d098f8.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=3712 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=de7095d9-fb61-40f7-b179-cdf2929bf838 EntryTimestamp=2021-08-11T16:11:58.369Z InvocationID=e25c164c-9f8b-4347-ae06-de410a7ddeb2 LogTimestamp=2021-08-11T16:12:02.081935Z ResponseDesc=Created Exiting. 2021-08-11T16:12:02.083Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:12:02.099Z [qtp2045766957-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=502a9f98-74e0-4ec8-aa63-4e3c665887ad InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:12:02.098682Z InvocationID=bcca1c2d-8e3c-48f7-ba62-f8e671f22166 LogTimestamp=2021-08-11T16:12:02.098812Z Entering 2021-08-11T16:12:02.099Z [qtp2045766957-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=502a9f98-74e0-4ec8-aa63-4e3c665887ad InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:12:02.098682Z InvocationID=bcca1c2d-8e3c-48f7-ba62-f8e671f22166 LogTimestamp=2021-08-11T16:12:02.098812Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:12:02.371Z [qtp2045766957-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:02.099Z InvocationID=4efc1ca3-1275-432f-863f-867fe698300a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:02.371Z [qtp2045766957-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:02.099Z InvocationID=4efc1ca3-1275-432f-863f-867fe698300a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:02.371Z [qtp2045766957-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:02.099Z InvocationID=4efc1ca3-1275-432f-863f-867fe698300a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:02.371Z [qtp2045766957-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:02.099Z InvocationID=4efc1ca3-1275-432f-863f-867fe698300a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:02.385Z [qtp2045766957-20] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:02.099Z InvocationID=4efc1ca3-1275-432f-863f-867fe698300a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Extcp2 2021-08-11T16:12:03.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:12:03.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:12:03.636Z [qtp2045766957-20] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ExtCP2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "cbd1ba82-febc-478d-b2d9-d9aedb49a1e8" INVARIANT_UUID = "fb8d8547-d985-4c77-8379-eaf527c9ea2c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.v2.extCP" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=cbd1ba82-febc-478d-b2d9-d9aedb49a1e8 EntryTimestamp=2021-08-11T16:12:02.099Z InvocationID=4efc1ca3-1275-432f-863f-867fe698300a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:12:03.637Z [qtp2045766957-20] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ExtCP2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "cbd1ba82-febc-478d-b2d9-d9aedb49a1e8" INVARIANT_UUID = "fb8d8547-d985-4c77-8379-eaf527c9ea2c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.v2.extCP" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=cbd1ba82-febc-478d-b2d9-d9aedb49a1e8 TargetServiceName=null EntryTimestamp=2021-08-11T16:12:02.099Z InvocationID=4efc1ca3-1275-432f-863f-867fe698300a ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:12:03.833Z [qtp2045766957-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ExtCP2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "cbd1ba82-febc-478d-b2d9-d9aedb49a1e8" INVARIANT_UUID = "fb8d8547-d985-4c77-8379-eaf527c9ea2c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.v2.extCP" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=cbd1ba82-febc-478d-b2d9-d9aedb49a1e8 TargetServiceName=null EntryTimestamp=2021-08-11T16:12:02.099Z InvocationID=4efc1ca3-1275-432f-863f-867fe698300a ErrorCode=500 lock resource with id 846ba3cc-17d7-4bed-af26-eabe968eb156 2021-08-11T16:12:04.259Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:12:04.608Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b6d9e36-0f55-4620-ba1a-46c6fa114126 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:12:04.608Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b6d9e36-0f55-4620-ba1a-46c6fa114126 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@72aaaec0 2021-08-11T16:12:04.608Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b6d9e36-0f55-4620-ba1a-46c6fa114126 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:12:04.608Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b6d9e36-0f55-4620-ba1a-46c6fa114126 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:12:05.460Z [qtp2045766957-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ExtCP2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "cbd1ba82-febc-478d-b2d9-d9aedb49a1e8" INVARIANT_UUID = "fb8d8547-d985-4c77-8379-eaf527c9ea2c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "846ba3cc-17d7-4bed-af26-eabe968eb156.assettoscatemplate" CURR_ARTIFACT_UUID = "846ba3cc-17d7-4bed-af26-eabe968eb156.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extcp2-template.yml,0,846ba3cc-17d7-4bed-af26-eabe968eb156.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=628447e2-0d86-41ec-ac94-6ea2695693df TargetServiceName=null EntryTimestamp=2021-08-11T16:12:02.099Z InvocationID=4efc1ca3-1275-432f-863f-867fe698300a ErrorCode=500 Starting to create Csar for component ExtCP2 2021-08-11T16:12:05.692Z [qtp2045766957-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ExtCP2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "cbd1ba82-febc-478d-b2d9-d9aedb49a1e8" INVARIANT_UUID = "fb8d8547-d985-4c77-8379-eaf527c9ea2c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "846ba3cc-17d7-4bed-af26-eabe968eb156.assettoscatemplate" CURR_ARTIFACT_UUID = "846ba3cc-17d7-4bed-af26-eabe968eb156.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extcp2-template.yml,0,846ba3cc-17d7-4bed-af26-eabe968eb156.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=628447e2-0d86-41ec-ac94-6ea2695693df TargetServiceName=null EntryTimestamp=2021-08-11T16:12:02.099Z InvocationID=4efc1ca3-1275-432f-863f-867fe698300a ErrorCode=500 Ended create Csar for component ExtCP2 2021-08-11T16:12:05.693Z [qtp2045766957-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ExtCP2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "cbd1ba82-febc-478d-b2d9-d9aedb49a1e8" INVARIANT_UUID = "fb8d8547-d985-4c77-8379-eaf527c9ea2c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "846ba3cc-17d7-4bed-af26-eabe968eb156.assettoscatemplate" CURR_ARTIFACT_UUID = "846ba3cc-17d7-4bed-af26-eabe968eb156.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extcp2-template.yml,0,846ba3cc-17d7-4bed-af26-eabe968eb156.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=628447e2-0d86-41ec-ac94-6ea2695693df TargetServiceName=null EntryTimestamp=2021-08-11T16:12:02.099Z InvocationID=4efc1ca3-1275-432f-863f-867fe698300a ErrorCode=500 lock resource with id 846ba3cc-17d7-4bed-af26-eabe968eb156 2021-08-11T16:12:07.316Z [qtp2045766957-20] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ExtCP2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "cbd1ba82-febc-478d-b2d9-d9aedb49a1e8" INVARIANT_UUID = "fb8d8547-d985-4c77-8379-eaf527c9ea2c" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "846ba3cc-17d7-4bed-af26-eabe968eb156.assettoscacsar" CURR_ARTIFACT_UUID = "846ba3cc-17d7-4bed-af26-eabe968eb156.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Extcp2-csar.csar,0,846ba3cc-17d7-4bed-af26-eabe968eb156.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5217 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b36d4f53-192b-4d1c-b4c5-1776fea57bf6 TargetServiceName=null EntryTimestamp=2021-08-11T16:12:02.099Z InvocationID=4efc1ca3-1275-432f-863f-867fe698300a ErrorCode=500 LogTimestamp=2021-08-11T16:12:07.316622Z ResponseDesc=Created Exiting. 2021-08-11T16:12:07.317Z [qtp2045766957-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:12:07.346Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=95199d4c-93b1-4a1f-b036-6b5725b89fc5 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:12:07.346586Z InvocationID=d18c8f35-a770-4850-b214-9f95babbfbae LogTimestamp=2021-08-11T16:12:07.346637Z Entering 2021-08-11T16:12:07.347Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=95199d4c-93b1-4a1f-b036-6b5725b89fc5 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:12:07.346586Z InvocationID=d18c8f35-a770-4850-b214-9f95babbfbae LogTimestamp=2021-08-11T16:12:07.346637Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:12:07.695Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=09b9a88d-aa70-45d2-9f7f-1ad818c925ab PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:12:07.695Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=09b9a88d-aa70-45d2-9f7f-1ad818c925ab PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6de15770 2021-08-11T16:12:07.695Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=09b9a88d-aa70-45d2-9f7f-1ad818c925ab PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:12:07.695Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=09b9a88d-aa70-45d2-9f7f-1ad818c925ab PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:12:07.854Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:07.347Z InvocationID=fb307188-6f61-4140-9219-3dd4bb5730f9 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:07.854Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:07.347Z InvocationID=fb307188-6f61-4140-9219-3dd4bb5730f9 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:07.854Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:07.347Z InvocationID=fb307188-6f61-4140-9219-3dd4bb5730f9 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:07.854Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:07.347Z InvocationID=fb307188-6f61-4140-9219-3dd4bb5730f9 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:07.882Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:07.347Z InvocationID=fb307188-6f61-4140-9219-3dd4bb5730f9 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Extneutroncp 2021-08-11T16:12:09.178Z [qtp2045766957-18] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "extNeutronCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "a5d24b84-8b5d-4bde-8c4b-6931a815ac0d" INVARIANT_UUID = "c2d41c5a-90ce-48ac-b51a-e9bb0c2053d6" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.v2.extNeutronCP" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a5d24b84-8b5d-4bde-8c4b-6931a815ac0d EntryTimestamp=2021-08-11T16:12:07.347Z InvocationID=fb307188-6f61-4140-9219-3dd4bb5730f9 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:12:09.179Z [qtp2045766957-18] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "extNeutronCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "a5d24b84-8b5d-4bde-8c4b-6931a815ac0d" INVARIANT_UUID = "c2d41c5a-90ce-48ac-b51a-e9bb0c2053d6" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.v2.extNeutronCP" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a5d24b84-8b5d-4bde-8c4b-6931a815ac0d TargetServiceName=null EntryTimestamp=2021-08-11T16:12:07.347Z InvocationID=fb307188-6f61-4140-9219-3dd4bb5730f9 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:12:09.408Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "extNeutronCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "a5d24b84-8b5d-4bde-8c4b-6931a815ac0d" INVARIANT_UUID = "c2d41c5a-90ce-48ac-b51a-e9bb0c2053d6" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.v2.extNeutronCP" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a5d24b84-8b5d-4bde-8c4b-6931a815ac0d TargetServiceName=null EntryTimestamp=2021-08-11T16:12:07.347Z InvocationID=fb307188-6f61-4140-9219-3dd4bb5730f9 ErrorCode=500 lock resource with id 719d1a47-5378-460f-9327-a0b5e33ba978 2021-08-11T16:12:10.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9e18d53-4511-4df0-9756-085baf891c91 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:12:10.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9e18d53-4511-4df0-9756-085baf891c91 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@404e556e 2021-08-11T16:12:10.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9e18d53-4511-4df0-9756-085baf891c91 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:12:10.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9e18d53-4511-4df0-9756-085baf891c91 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:12:11.004Z [qtp2045766957-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "extNeutronCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "a5d24b84-8b5d-4bde-8c4b-6931a815ac0d" INVARIANT_UUID = "c2d41c5a-90ce-48ac-b51a-e9bb0c2053d6" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "719d1a47-5378-460f-9327-a0b5e33ba978.assettoscatemplate" CURR_ARTIFACT_UUID = "719d1a47-5378-460f-9327-a0b5e33ba978.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extneutroncp-template.yml,0,719d1a47-5378-460f-9327-a0b5e33ba978.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f4c4b923-7973-4537-8ee7-87d10278288f TargetServiceName=null EntryTimestamp=2021-08-11T16:12:07.347Z InvocationID=fb307188-6f61-4140-9219-3dd4bb5730f9 ErrorCode=500 Starting to create Csar for component extNeutronCP 2021-08-11T16:12:11.198Z [qtp2045766957-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "extNeutronCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "a5d24b84-8b5d-4bde-8c4b-6931a815ac0d" INVARIANT_UUID = "c2d41c5a-90ce-48ac-b51a-e9bb0c2053d6" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "719d1a47-5378-460f-9327-a0b5e33ba978.assettoscatemplate" CURR_ARTIFACT_UUID = "719d1a47-5378-460f-9327-a0b5e33ba978.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extneutroncp-template.yml,0,719d1a47-5378-460f-9327-a0b5e33ba978.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f4c4b923-7973-4537-8ee7-87d10278288f TargetServiceName=null EntryTimestamp=2021-08-11T16:12:07.347Z InvocationID=fb307188-6f61-4140-9219-3dd4bb5730f9 ErrorCode=500 Ended create Csar for component extNeutronCP 2021-08-11T16:12:11.199Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "extNeutronCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "a5d24b84-8b5d-4bde-8c4b-6931a815ac0d" INVARIANT_UUID = "c2d41c5a-90ce-48ac-b51a-e9bb0c2053d6" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "719d1a47-5378-460f-9327-a0b5e33ba978.assettoscatemplate" CURR_ARTIFACT_UUID = "719d1a47-5378-460f-9327-a0b5e33ba978.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extneutroncp-template.yml,0,719d1a47-5378-460f-9327-a0b5e33ba978.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f4c4b923-7973-4537-8ee7-87d10278288f TargetServiceName=null EntryTimestamp=2021-08-11T16:12:07.347Z InvocationID=fb307188-6f61-4140-9219-3dd4bb5730f9 ErrorCode=500 lock resource with id 719d1a47-5378-460f-9327-a0b5e33ba978 2021-08-11T16:12:12.831Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "extNeutronCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "a5d24b84-8b5d-4bde-8c4b-6931a815ac0d" INVARIANT_UUID = "c2d41c5a-90ce-48ac-b51a-e9bb0c2053d6" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "719d1a47-5378-460f-9327-a0b5e33ba978.assettoscacsar" CURR_ARTIFACT_UUID = "719d1a47-5378-460f-9327-a0b5e33ba978.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Extneutroncp-csar.csar,0,719d1a47-5378-460f-9327-a0b5e33ba978.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5484 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=219c24f1-6290-4627-b3b6-4747ee9c4c9e TargetServiceName=null EntryTimestamp=2021-08-11T16:12:07.347Z InvocationID=fb307188-6f61-4140-9219-3dd4bb5730f9 ErrorCode=500 LogTimestamp=2021-08-11T16:12:12.831312Z ResponseDesc=Created Exiting. 2021-08-11T16:12:12.832Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:12:12.849Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=cd01b7a9-e03f-492b-8e1b-ee3f5cdceff4 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:12:12.848906Z InvocationID=0975f462-8910-4ecc-bafb-1ed2994aab68 LogTimestamp=2021-08-11T16:12:12.848945Z Entering 2021-08-11T16:12:12.849Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=cd01b7a9-e03f-492b-8e1b-ee3f5cdceff4 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:12:12.848906Z InvocationID=0975f462-8910-4ecc-bafb-1ed2994aab68 LogTimestamp=2021-08-11T16:12:12.848945Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:12:13.172Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:12.849Z InvocationID=8e8f8c5d-db9e-4e32-8a70-e597ecc7f343 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:13.173Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:12.849Z InvocationID=8e8f8c5d-db9e-4e32-8a70-e597ecc7f343 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:13.173Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:12.849Z InvocationID=8e8f8c5d-db9e-4e32-8a70-e597ecc7f343 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:13.173Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:12.849Z InvocationID=8e8f8c5d-db9e-4e32-8a70-e597ecc7f343 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:13.173Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:12.849Z InvocationID=8e8f8c5d-db9e-4e32-8a70-e597ecc7f343 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Extcontrailcp 2021-08-11T16:12:13.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c09225f8-4946-4458-ae01-98a97efcbbc8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:12:13.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c09225f8-4946-4458-ae01-98a97efcbbc8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6bc65d1 2021-08-11T16:12:13.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c09225f8-4946-4458-ae01-98a97efcbbc8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:12:13.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c09225f8-4946-4458-ae01-98a97efcbbc8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:12:14.586Z [qtp2045766957-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "extContrailCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "82a377cb-1d2b-43a3-baaf-fe0165e07ab2" INVARIANT_UUID = "39c834e1-317e-4548-8d9a-2d5a6837ac31" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.v2.extContrailCP" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=82a377cb-1d2b-43a3-baaf-fe0165e07ab2 EntryTimestamp=2021-08-11T16:12:12.849Z InvocationID=8e8f8c5d-db9e-4e32-8a70-e597ecc7f343 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:12:14.587Z [qtp2045766957-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "extContrailCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "82a377cb-1d2b-43a3-baaf-fe0165e07ab2" INVARIANT_UUID = "39c834e1-317e-4548-8d9a-2d5a6837ac31" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.v2.extContrailCP" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=82a377cb-1d2b-43a3-baaf-fe0165e07ab2 TargetServiceName=null EntryTimestamp=2021-08-11T16:12:12.849Z InvocationID=8e8f8c5d-db9e-4e32-8a70-e597ecc7f343 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:12:14.861Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "extContrailCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "82a377cb-1d2b-43a3-baaf-fe0165e07ab2" INVARIANT_UUID = "39c834e1-317e-4548-8d9a-2d5a6837ac31" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.v2.extContrailCP" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=82a377cb-1d2b-43a3-baaf-fe0165e07ab2 TargetServiceName=null EntryTimestamp=2021-08-11T16:12:12.849Z InvocationID=8e8f8c5d-db9e-4e32-8a70-e597ecc7f343 ErrorCode=500 lock resource with id 8c70a69e-632b-4c9e-a6ef-7204fdeba6fc 2021-08-11T16:12:16.573Z [qtp2045766957-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "extContrailCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "82a377cb-1d2b-43a3-baaf-fe0165e07ab2" INVARIANT_UUID = "39c834e1-317e-4548-8d9a-2d5a6837ac31" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "8c70a69e-632b-4c9e-a6ef-7204fdeba6fc.assettoscatemplate" CURR_ARTIFACT_UUID = "8c70a69e-632b-4c9e-a6ef-7204fdeba6fc.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extcontrailcp-template.yml,0,8c70a69e-632b-4c9e-a6ef-7204fdeba6fc.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=29b2b818-678b-4eb7-aff8-3c17cf9e6b13 TargetServiceName=null EntryTimestamp=2021-08-11T16:12:12.849Z InvocationID=8e8f8c5d-db9e-4e32-8a70-e597ecc7f343 ErrorCode=500 Starting to create Csar for component extContrailCP 2021-08-11T16:12:16.704Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ca8b599-b105-4ea9-83f5-b5b5c3e969a1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:12:16.704Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ca8b599-b105-4ea9-83f5-b5b5c3e969a1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@21c71768 2021-08-11T16:12:16.704Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ca8b599-b105-4ea9-83f5-b5b5c3e969a1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:12:16.704Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ca8b599-b105-4ea9-83f5-b5b5c3e969a1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:12:16.777Z [qtp2045766957-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "extContrailCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "82a377cb-1d2b-43a3-baaf-fe0165e07ab2" INVARIANT_UUID = "39c834e1-317e-4548-8d9a-2d5a6837ac31" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "8c70a69e-632b-4c9e-a6ef-7204fdeba6fc.assettoscatemplate" CURR_ARTIFACT_UUID = "8c70a69e-632b-4c9e-a6ef-7204fdeba6fc.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extcontrailcp-template.yml,0,8c70a69e-632b-4c9e-a6ef-7204fdeba6fc.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=29b2b818-678b-4eb7-aff8-3c17cf9e6b13 TargetServiceName=null EntryTimestamp=2021-08-11T16:12:12.849Z InvocationID=8e8f8c5d-db9e-4e32-8a70-e597ecc7f343 ErrorCode=500 Ended create Csar for component extContrailCP 2021-08-11T16:12:16.778Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "extContrailCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "82a377cb-1d2b-43a3-baaf-fe0165e07ab2" INVARIANT_UUID = "39c834e1-317e-4548-8d9a-2d5a6837ac31" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "8c70a69e-632b-4c9e-a6ef-7204fdeba6fc.assettoscatemplate" CURR_ARTIFACT_UUID = "8c70a69e-632b-4c9e-a6ef-7204fdeba6fc.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extcontrailcp-template.yml,0,8c70a69e-632b-4c9e-a6ef-7204fdeba6fc.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=29b2b818-678b-4eb7-aff8-3c17cf9e6b13 TargetServiceName=null EntryTimestamp=2021-08-11T16:12:12.849Z InvocationID=8e8f8c5d-db9e-4e32-8a70-e597ecc7f343 ErrorCode=500 lock resource with id 8c70a69e-632b-4c9e-a6ef-7204fdeba6fc 2021-08-11T16:12:18.305Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:12:18.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:12:18.395Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "extContrailCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "82a377cb-1d2b-43a3-baaf-fe0165e07ab2" INVARIANT_UUID = "39c834e1-317e-4548-8d9a-2d5a6837ac31" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "8c70a69e-632b-4c9e-a6ef-7204fdeba6fc.assettoscacsar" CURR_ARTIFACT_UUID = "8c70a69e-632b-4c9e-a6ef-7204fdeba6fc.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Extcontrailcp-csar.csar,0,8c70a69e-632b-4c9e-a6ef-7204fdeba6fc.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5546 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9fdbe329-9278-474b-9069-7d96cb4bba57 TargetServiceName=null EntryTimestamp=2021-08-11T16:12:12.849Z InvocationID=8e8f8c5d-db9e-4e32-8a70-e597ecc7f343 ErrorCode=500 LogTimestamp=2021-08-11T16:12:18.395175Z ResponseDesc=Created Exiting. 2021-08-11T16:12:18.395Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:12:18.413Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=6f8efd44-cdcc-4201-9f51-78928d97702f InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:12:18.413388Z InvocationID=8e598eed-2ed4-403c-806d-6b3c99d8fbd8 LogTimestamp=2021-08-11T16:12:18.413440Z Entering 2021-08-11T16:12:18.413Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=6f8efd44-cdcc-4201-9f51-78928d97702f InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:12:18.413388Z InvocationID=8e598eed-2ed4-403c-806d-6b3c99d8fbd8 LogTimestamp=2021-08-11T16:12:18.413440Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:12:18.744Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:18.414Z InvocationID=49c10829-dfe6-4dd3-a51e-67f4f5907ba3 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:18.744Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:18.414Z InvocationID=49c10829-dfe6-4dd3-a51e-67f4f5907ba3 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:18.744Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:18.414Z InvocationID=49c10829-dfe6-4dd3-a51e-67f4f5907ba3 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:18.744Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:18.414Z InvocationID=49c10829-dfe6-4dd3-a51e-67f4f5907ba3 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:18.763Z [qtp2045766957-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:18.414Z InvocationID=49c10829-dfe6-4dd3-a51e-67f4f5907ba3 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name PortMirroringConfigurationByPolicy 2021-08-11T16:12:19.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6b54f8f6-6e4c-481f-a87f-ac06a0eca96b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:12:19.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6b54f8f6-6e4c-481f-a87f-ac06a0eca96b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@467a8ef7 2021-08-11T16:12:19.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6b54f8f6-6e4c-481f-a87f-ac06a0eca96b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:12:19.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6b54f8f6-6e4c-481f-a87f-ac06a0eca96b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:12:20.242Z [qtp2045766957-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Port Mirroring Configuration By Policy" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "ecc44433-e610-4160-bf6f-bfc2de24f4be" INVARIANT_UUID = "251c9f47-62bc-4287-99bb-a1e6cbdd3f25" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.nodes.PortMirroringConfigurationByPolicy" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecc44433-e610-4160-bf6f-bfc2de24f4be EntryTimestamp=2021-08-11T16:12:18.414Z InvocationID=49c10829-dfe6-4dd3-a51e-67f4f5907ba3 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id 2765b36c-c65d-482b-92e0-c3876ae1826f 2021-08-11T16:12:21.863Z [qtp2045766957-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Port Mirroring Configuration By Policy" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "ecc44433-e610-4160-bf6f-bfc2de24f4be" INVARIANT_UUID = "251c9f47-62bc-4287-99bb-a1e6cbdd3f25" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "2765b36c-c65d-482b-92e0-c3876ae1826f.assettoscatemplate" CURR_ARTIFACT_UUID = "2765b36c-c65d-482b-92e0-c3876ae1826f.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-PortMirroringConfigurationByPolicy-template.yml,0,2765b36c-c65d-482b-92e0-c3876ae1826f.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c2e4e2e5-126d-4bb7-8763-3144eb1b3314 EntryTimestamp=2021-08-11T16:12:18.414Z InvocationID=49c10829-dfe6-4dd3-a51e-67f4f5907ba3 PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Starting to create Csar for component Port Mirroring Configuration By Policy 2021-08-11T16:12:22.051Z [qtp2045766957-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Port Mirroring Configuration By Policy" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "ecc44433-e610-4160-bf6f-bfc2de24f4be" INVARIANT_UUID = "251c9f47-62bc-4287-99bb-a1e6cbdd3f25" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "2765b36c-c65d-482b-92e0-c3876ae1826f.assettoscatemplate" CURR_ARTIFACT_UUID = "2765b36c-c65d-482b-92e0-c3876ae1826f.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-PortMirroringConfigurationByPolicy-template.yml,0,2765b36c-c65d-482b-92e0-c3876ae1826f.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c2e4e2e5-126d-4bb7-8763-3144eb1b3314 EntryTimestamp=2021-08-11T16:12:18.414Z InvocationID=49c10829-dfe6-4dd3-a51e-67f4f5907ba3 Ended create Csar for component Port Mirroring Configuration By Policy 2021-08-11T16:12:22.052Z [qtp2045766957-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Port Mirroring Configuration By Policy" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "ecc44433-e610-4160-bf6f-bfc2de24f4be" INVARIANT_UUID = "251c9f47-62bc-4287-99bb-a1e6cbdd3f25" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "2765b36c-c65d-482b-92e0-c3876ae1826f.assettoscatemplate" CURR_ARTIFACT_UUID = "2765b36c-c65d-482b-92e0-c3876ae1826f.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-PortMirroringConfigurationByPolicy-template.yml,0,2765b36c-c65d-482b-92e0-c3876ae1826f.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c2e4e2e5-126d-4bb7-8763-3144eb1b3314 EntryTimestamp=2021-08-11T16:12:18.414Z InvocationID=49c10829-dfe6-4dd3-a51e-67f4f5907ba3 lock resource with id 2765b36c-c65d-482b-92e0-c3876ae1826f 2021-08-11T16:12:22.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24e49559-334a-4ead-9c98-cb85d8689df4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:12:22.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24e49559-334a-4ead-9c98-cb85d8689df4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@742d4673 2021-08-11T16:12:22.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24e49559-334a-4ead-9c98-cb85d8689df4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:12:22.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24e49559-334a-4ead-9c98-cb85d8689df4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:12:23.672Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Port Mirroring Configuration By Policy" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "ecc44433-e610-4160-bf6f-bfc2de24f4be" INVARIANT_UUID = "251c9f47-62bc-4287-99bb-a1e6cbdd3f25" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "2765b36c-c65d-482b-92e0-c3876ae1826f.assettoscacsar" CURR_ARTIFACT_UUID = "2765b36c-c65d-482b-92e0-c3876ae1826f.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-PortMirroringConfigurationByPolicy-csar.csar,0,2765b36c-c65d-482b-92e0-c3876ae1826f.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5258 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=dca51c05-dee0-425f-847f-79ec93940a69 EntryTimestamp=2021-08-11T16:12:18.414Z InvocationID=49c10829-dfe6-4dd3-a51e-67f4f5907ba3 LogTimestamp=2021-08-11T16:12:23.672411Z ResponseDesc=Created Exiting. 2021-08-11T16:12:23.673Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:12:23.688Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=5719933b-96d0-402e-9648-0c3e2304c9d9 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:12:23.688312Z InvocationID=f245eb4e-c242-49e6-8442-d632f02db103 LogTimestamp=2021-08-11T16:12:23.688351Z Entering 2021-08-11T16:12:23.688Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=5719933b-96d0-402e-9648-0c3e2304c9d9 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:12:23.688312Z InvocationID=f245eb4e-c242-49e6-8442-d632f02db103 LogTimestamp=2021-08-11T16:12:23.688351Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:12:24.254Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:23.688Z InvocationID=e409cd13-9a02-4525-a3c3-fa6e6b651d34 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:24.254Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:23.688Z InvocationID=e409cd13-9a02-4525-a3c3-fa6e6b651d34 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:24.254Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:23.688Z InvocationID=e409cd13-9a02-4525-a3c3-fa6e6b651d34 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:24.254Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:23.688Z InvocationID=e409cd13-9a02-4525-a3c3-fa6e6b651d34 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:24.268Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:23.688Z InvocationID=e409cd13-9a02-4525-a3c3-fa6e6b651d34 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name ForwardingPath 2021-08-11T16:12:25.750Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=340f3e80-0976-4d26-9f31-0fac9bf50cf8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:12:25.750Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=340f3e80-0976-4d26-9f31-0fac9bf50cf8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@115a27ca 2021-08-11T16:12:25.750Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=340f3e80-0976-4d26-9f31-0fac9bf50cf8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:12:25.750Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=340f3e80-0976-4d26-9f31-0fac9bf50cf8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:12:26.672Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Forwarding Path" RESOURCE_TYPE = "ABSTRACT" SERVICE_INSTANCE_ID = "9d29bf36-1669-4d05-a5c9-a5bdca23f442" INVARIANT_UUID = "7fc404c3-f530-4938-869f-be52fa3e233d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.nodes.ForwardingPath" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9d29bf36-1669-4d05-a5c9-a5bdca23f442 EntryTimestamp=2021-08-11T16:12:23.688Z InvocationID=e409cd13-9a02-4525-a3c3-fa6e6b651d34 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id e7b85d23-bc7b-439e-9617-ad7a04f2ebf9 2021-08-11T16:12:28.454Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Forwarding Path" RESOURCE_TYPE = "ABSTRACT" SERVICE_INSTANCE_ID = "9d29bf36-1669-4d05-a5c9-a5bdca23f442" INVARIANT_UUID = "7fc404c3-f530-4938-869f-be52fa3e233d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "e7b85d23-bc7b-439e-9617-ad7a04f2ebf9.assettoscatemplate" CURR_ARTIFACT_UUID = "e7b85d23-bc7b-439e-9617-ad7a04f2ebf9.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-ForwardingPath-template.yml,0,e7b85d23-bc7b-439e-9617-ad7a04f2ebf9.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=4766 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=182971d5-96f4-46c0-8e02-cb2db00b0612 EntryTimestamp=2021-08-11T16:12:23.688Z InvocationID=e409cd13-9a02-4525-a3c3-fa6e6b651d34 LogTimestamp=2021-08-11T16:12:28.454513Z ResponseDesc=Created Exiting. 2021-08-11T16:12:28.455Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:12:28.468Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=8ea9b8f8-b015-4ef3-afec-b6e2deb6af6c InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:12:28.468572Z InvocationID=aa7ffc0a-1c01-4d3b-815a-62b3590a37be LogTimestamp=2021-08-11T16:12:28.468625Z Entering 2021-08-11T16:12:28.469Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=8ea9b8f8-b015-4ef3-afec-b6e2deb6af6c InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:12:28.468572Z InvocationID=aa7ffc0a-1c01-4d3b-815a-62b3590a37be LogTimestamp=2021-08-11T16:12:28.468625Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:12:28.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2a5ea0a-d242-49f4-94c6-bacee704ce60 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:12:28.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2a5ea0a-d242-49f4-94c6-bacee704ce60 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@8bf0ede 2021-08-11T16:12:28.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2a5ea0a-d242-49f4-94c6-bacee704ce60 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:12:28.959Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2a5ea0a-d242-49f4-94c6-bacee704ce60 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:12:30.050Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:28.469Z InvocationID=4b3e8a35-5d96-4df2-8f33-1d57061c8505 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:30.050Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:28.469Z InvocationID=4b3e8a35-5d96-4df2-8f33-1d57061c8505 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:30.050Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:28.469Z InvocationID=4b3e8a35-5d96-4df2-8f33-1d57061c8505 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:30.050Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:28.469Z InvocationID=4b3e8a35-5d96-4df2-8f33-1d57061c8505 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:30.051Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:28.469Z InvocationID=4b3e8a35-5d96-4df2-8f33-1d57061c8505 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Configuration 2021-08-11T16:12:31.578Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Configuration" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "1e3a8823-9af4-472e-8ea3-5214725672f6" INVARIANT_UUID = "40f9106b-828d-4181-ab99-77b17223a573" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.nodes.Configuration" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1e3a8823-9af4-472e-8ea3-5214725672f6 EntryTimestamp=2021-08-11T16:12:28.469Z InvocationID=4b3e8a35-5d96-4df2-8f33-1d57061c8505 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id c0629093-2c59-4d3b-96a5-ea1033962f86 2021-08-11T16:12:31.716Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d400c754-70ae-4617-a2dc-c4e2759fd6ae PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:12:31.717Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d400c754-70ae-4617-a2dc-c4e2759fd6ae PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1a202408 2021-08-11T16:12:31.717Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d400c754-70ae-4617-a2dc-c4e2759fd6ae PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:12:31.717Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d400c754-70ae-4617-a2dc-c4e2759fd6ae PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:12:33.181Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Configuration" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "1e3a8823-9af4-472e-8ea3-5214725672f6" INVARIANT_UUID = "40f9106b-828d-4181-ab99-77b17223a573" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c0629093-2c59-4d3b-96a5-ea1033962f86.assettoscatemplate" CURR_ARTIFACT_UUID = "c0629093-2c59-4d3b-96a5-ea1033962f86.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Configuration-template.yml,0,c0629093-2c59-4d3b-96a5-ea1033962f86.assettoscatemplate,0 " STATUS = "200" DESC = "OK" ResponseCode=201 RequestID=null ElapsedTime=4712 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=469fb358-fbbf-48aa-b9a6-1c8eb3ec6414 EntryTimestamp=2021-08-11T16:12:28.469Z InvocationID=4b3e8a35-5d96-4df2-8f33-1d57061c8505 LogTimestamp=2021-08-11T16:12:33.181257Z ResponseDesc=Created Exiting. 2021-08-11T16:12:33.181Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:12:33.195Z [qtp2045766957-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=30cf2c3d-4f51-4bd9-bb85-ad548c47260a InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:12:33.195576Z InvocationID=02ccff95-c8d2-48f1-9e77-c9a942db699a LogTimestamp=2021-08-11T16:12:33.195618Z Entering 2021-08-11T16:12:33.196Z [qtp2045766957-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=30cf2c3d-4f51-4bd9-bb85-ad548c47260a InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:12:33.195576Z InvocationID=02ccff95-c8d2-48f1-9e77-c9a942db699a LogTimestamp=2021-08-11T16:12:33.195618Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:12:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:12:33.316Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:12:33.485Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:33.196Z InvocationID=e4dfb1d6-5ac6-4e8d-ba68-92c1be37ba39 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:33.485Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:33.196Z InvocationID=e4dfb1d6-5ac6-4e8d-ba68-92c1be37ba39 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:33.485Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:33.196Z InvocationID=e4dfb1d6-5ac6-4e8d-ba68-92c1be37ba39 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:33.485Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:33.196Z InvocationID=e4dfb1d6-5ac6-4e8d-ba68-92c1be37ba39 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:33.498Z [qtp2045766957-11] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:33.196Z InvocationID=e4dfb1d6-5ac6-4e8d-ba68-92c1be37ba39 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name VrfObjectConfiguration 2021-08-11T16:12:34.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3a5c7ad-b784-4678-a4ab-cd5ea8aa62f0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:12:34.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3a5c7ad-b784-4678-a4ab-cd5ea8aa62f0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4a0e7e1 2021-08-11T16:12:34.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3a5c7ad-b784-4678-a4ab-cd5ea8aa62f0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:12:34.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3a5c7ad-b784-4678-a4ab-cd5ea8aa62f0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:12:35.181Z [qtp2045766957-11] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VRF Object Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "edaee223-0908-4c62-9be6-a7b4bfe3b39d" INVARIANT_UUID = "86472496-01dc-4f59-a3b8-4312e81a2fbb" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.nodes.VRFObject" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=edaee223-0908-4c62-9be6-a7b4bfe3b39d EntryTimestamp=2021-08-11T16:12:33.196Z InvocationID=e4dfb1d6-5ac6-4e8d-ba68-92c1be37ba39 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id 86df24bc-2239-4d02-a154-66ff6596e4b1 2021-08-11T16:12:36.780Z [qtp2045766957-11] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Object Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "edaee223-0908-4c62-9be6-a7b4bfe3b39d" INVARIANT_UUID = "86472496-01dc-4f59-a3b8-4312e81a2fbb" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "86df24bc-2239-4d02-a154-66ff6596e4b1.assettoscatemplate" CURR_ARTIFACT_UUID = "86df24bc-2239-4d02-a154-66ff6596e4b1.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfObjectConfiguration-template.yml,0,86df24bc-2239-4d02-a154-66ff6596e4b1.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=4ef4c2b8-2ba4-4559-98d4-77b4f8c4dc59 EntryTimestamp=2021-08-11T16:12:33.196Z InvocationID=e4dfb1d6-5ac6-4e8d-ba68-92c1be37ba39 PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Starting to create Csar for component VRF Object Configuration 2021-08-11T16:12:36.994Z [qtp2045766957-11] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Object Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "edaee223-0908-4c62-9be6-a7b4bfe3b39d" INVARIANT_UUID = "86472496-01dc-4f59-a3b8-4312e81a2fbb" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "86df24bc-2239-4d02-a154-66ff6596e4b1.assettoscatemplate" CURR_ARTIFACT_UUID = "86df24bc-2239-4d02-a154-66ff6596e4b1.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfObjectConfiguration-template.yml,0,86df24bc-2239-4d02-a154-66ff6596e4b1.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=4ef4c2b8-2ba4-4559-98d4-77b4f8c4dc59 EntryTimestamp=2021-08-11T16:12:33.196Z InvocationID=e4dfb1d6-5ac6-4e8d-ba68-92c1be37ba39 Ended create Csar for component VRF Object Configuration 2021-08-11T16:12:36.994Z [qtp2045766957-11] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Object Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "edaee223-0908-4c62-9be6-a7b4bfe3b39d" INVARIANT_UUID = "86472496-01dc-4f59-a3b8-4312e81a2fbb" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "86df24bc-2239-4d02-a154-66ff6596e4b1.assettoscatemplate" CURR_ARTIFACT_UUID = "86df24bc-2239-4d02-a154-66ff6596e4b1.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfObjectConfiguration-template.yml,0,86df24bc-2239-4d02-a154-66ff6596e4b1.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=4ef4c2b8-2ba4-4559-98d4-77b4f8c4dc59 EntryTimestamp=2021-08-11T16:12:33.196Z InvocationID=e4dfb1d6-5ac6-4e8d-ba68-92c1be37ba39 lock resource with id 86df24bc-2239-4d02-a154-66ff6596e4b1 2021-08-11T16:12:37.721Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9960909c-0138-44b1-94b8-5e18a975c3ac PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:12:37.721Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9960909c-0138-44b1-94b8-5e18a975c3ac PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4404e4c8 2021-08-11T16:12:37.721Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9960909c-0138-44b1-94b8-5e18a975c3ac PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:12:37.721Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9960909c-0138-44b1-94b8-5e18a975c3ac PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:12:38.665Z [qtp2045766957-11] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Object Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "edaee223-0908-4c62-9be6-a7b4bfe3b39d" INVARIANT_UUID = "86472496-01dc-4f59-a3b8-4312e81a2fbb" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "86df24bc-2239-4d02-a154-66ff6596e4b1.assettoscacsar" CURR_ARTIFACT_UUID = "86df24bc-2239-4d02-a154-66ff6596e4b1.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VrfObjectConfiguration-csar.csar,0,86df24bc-2239-4d02-a154-66ff6596e4b1.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5469 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=962e2620-67df-4f1a-91c3-b4fa1c087624 EntryTimestamp=2021-08-11T16:12:33.196Z InvocationID=e4dfb1d6-5ac6-4e8d-ba68-92c1be37ba39 LogTimestamp=2021-08-11T16:12:38.665511Z ResponseDesc=Created Exiting. 2021-08-11T16:12:38.666Z [qtp2045766957-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:12:38.684Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=422ec6ad-fb7e-4af4-895f-1821f8358f41 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:12:38.683809Z InvocationID=63f39a4e-ffbe-4017-a0bf-4fc60ed7f7aa LogTimestamp=2021-08-11T16:12:38.683869Z Entering 2021-08-11T16:12:38.684Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=422ec6ad-fb7e-4af4-895f-1821f8358f41 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:12:38.683809Z InvocationID=63f39a4e-ffbe-4017-a0bf-4fc60ed7f7aa LogTimestamp=2021-08-11T16:12:38.683869Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:12:39.013Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:38.684Z InvocationID=8a792b02-5c70-43a5-9ca3-8a72d5c328b3 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:39.013Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:38.684Z InvocationID=8a792b02-5c70-43a5-9ca3-8a72d5c328b3 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:39.013Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:38.684Z InvocationID=8a792b02-5c70-43a5-9ca3-8a72d5c328b3 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:39.013Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:38.684Z InvocationID=8a792b02-5c70-43a5-9ca3-8a72d5c328b3 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:39.025Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:38.684Z InvocationID=8a792b02-5c70-43a5-9ca3-8a72d5c328b3 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Extvirtualmachineinterfacecp 2021-08-11T16:12:40.292Z [qtp2045766957-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "extVirtualMachineInterfaceCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "7e3c8440-21cf-44ee-8fd6-42f7c107a954" INVARIANT_UUID = "44585248-f87d-4420-a7f8-e4a5a41480b1" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.v2.extVirtualMachineInterfaceCP" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7e3c8440-21cf-44ee-8fd6-42f7c107a954 EntryTimestamp=2021-08-11T16:12:38.684Z InvocationID=8a792b02-5c70-43a5-9ca3-8a72d5c328b3 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:12:40.292Z [qtp2045766957-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "extVirtualMachineInterfaceCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "7e3c8440-21cf-44ee-8fd6-42f7c107a954" INVARIANT_UUID = "44585248-f87d-4420-a7f8-e4a5a41480b1" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.v2.extVirtualMachineInterfaceCP" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7e3c8440-21cf-44ee-8fd6-42f7c107a954 TargetServiceName=null EntryTimestamp=2021-08-11T16:12:38.684Z InvocationID=8a792b02-5c70-43a5-9ca3-8a72d5c328b3 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:12:40.477Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "extVirtualMachineInterfaceCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "7e3c8440-21cf-44ee-8fd6-42f7c107a954" INVARIANT_UUID = "44585248-f87d-4420-a7f8-e4a5a41480b1" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.v2.extVirtualMachineInterfaceCP" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7e3c8440-21cf-44ee-8fd6-42f7c107a954 TargetServiceName=null EntryTimestamp=2021-08-11T16:12:38.684Z InvocationID=8a792b02-5c70-43a5-9ca3-8a72d5c328b3 ErrorCode=500 lock resource with id f9a80afb-1374-45fa-9f04-8972828382be 2021-08-11T16:12:40.603Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=169aee44-943f-4454-8fde-b54765c28974 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:12:40.603Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=169aee44-943f-4454-8fde-b54765c28974 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@34bddb68 2021-08-11T16:12:40.603Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=169aee44-943f-4454-8fde-b54765c28974 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:12:40.603Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=169aee44-943f-4454-8fde-b54765c28974 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:12:42.091Z [qtp2045766957-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "extVirtualMachineInterfaceCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "7e3c8440-21cf-44ee-8fd6-42f7c107a954" INVARIANT_UUID = "44585248-f87d-4420-a7f8-e4a5a41480b1" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "f9a80afb-1374-45fa-9f04-8972828382be.assettoscatemplate" CURR_ARTIFACT_UUID = "f9a80afb-1374-45fa-9f04-8972828382be.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extvirtualmachineinterfacecp-template.yml,0,f9a80afb-1374-45fa-9f04-8972828382be.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=48262436-9a73-471d-971a-0451bf5716b8 TargetServiceName=null EntryTimestamp=2021-08-11T16:12:38.684Z InvocationID=8a792b02-5c70-43a5-9ca3-8a72d5c328b3 ErrorCode=500 Starting to create Csar for component extVirtualMachineInterfaceCP 2021-08-11T16:12:42.316Z [qtp2045766957-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "extVirtualMachineInterfaceCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "7e3c8440-21cf-44ee-8fd6-42f7c107a954" INVARIANT_UUID = "44585248-f87d-4420-a7f8-e4a5a41480b1" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "f9a80afb-1374-45fa-9f04-8972828382be.assettoscatemplate" CURR_ARTIFACT_UUID = "f9a80afb-1374-45fa-9f04-8972828382be.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extvirtualmachineinterfacecp-template.yml,0,f9a80afb-1374-45fa-9f04-8972828382be.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=48262436-9a73-471d-971a-0451bf5716b8 TargetServiceName=null EntryTimestamp=2021-08-11T16:12:38.684Z InvocationID=8a792b02-5c70-43a5-9ca3-8a72d5c328b3 ErrorCode=500 Ended create Csar for component extVirtualMachineInterfaceCP 2021-08-11T16:12:42.316Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "extVirtualMachineInterfaceCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "7e3c8440-21cf-44ee-8fd6-42f7c107a954" INVARIANT_UUID = "44585248-f87d-4420-a7f8-e4a5a41480b1" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "f9a80afb-1374-45fa-9f04-8972828382be.assettoscatemplate" CURR_ARTIFACT_UUID = "f9a80afb-1374-45fa-9f04-8972828382be.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Extvirtualmachineinterfacecp-template.yml,0,f9a80afb-1374-45fa-9f04-8972828382be.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=48262436-9a73-471d-971a-0451bf5716b8 TargetServiceName=null EntryTimestamp=2021-08-11T16:12:38.684Z InvocationID=8a792b02-5c70-43a5-9ca3-8a72d5c328b3 ErrorCode=500 lock resource with id f9a80afb-1374-45fa-9f04-8972828382be 2021-08-11T16:12:43.633Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=05821b76-44ce-44d4-91f3-8eda0de5a83b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:12:43.633Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=05821b76-44ce-44d4-91f3-8eda0de5a83b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2fa6b993 2021-08-11T16:12:43.633Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=05821b76-44ce-44d4-91f3-8eda0de5a83b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:12:43.633Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=05821b76-44ce-44d4-91f3-8eda0de5a83b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:12:43.929Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "extVirtualMachineInterfaceCP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "7e3c8440-21cf-44ee-8fd6-42f7c107a954" INVARIANT_UUID = "44585248-f87d-4420-a7f8-e4a5a41480b1" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "f9a80afb-1374-45fa-9f04-8972828382be.assettoscacsar" CURR_ARTIFACT_UUID = "f9a80afb-1374-45fa-9f04-8972828382be.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Extvirtualmachineinterfacecp-csar.csar,0,f9a80afb-1374-45fa-9f04-8972828382be.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5245 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=8061132a-30c6-4cff-8cdc-b868cedd3567 TargetServiceName=null EntryTimestamp=2021-08-11T16:12:38.684Z InvocationID=8a792b02-5c70-43a5-9ca3-8a72d5c328b3 ErrorCode=500 LogTimestamp=2021-08-11T16:12:43.929311Z ResponseDesc=Created Exiting. 2021-08-11T16:12:43.929Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:12:43.943Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=fbfc87f5-d9ee-4e24-8093-56ef31ada43e InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:12:43.943061Z InvocationID=6d1d339b-d079-4dc4-b9e5-e3f19f276bd4 LogTimestamp=2021-08-11T16:12:43.943127Z Entering 2021-08-11T16:12:43.943Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=fbfc87f5-d9ee-4e24-8093-56ef31ada43e InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:12:43.943061Z InvocationID=6d1d339b-d079-4dc4-b9e5-e3f19f276bd4 LogTimestamp=2021-08-11T16:12:43.943127Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:12:44.266Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:43.943Z InvocationID=1039aa4a-0fb4-4713-86c4-d64c832cc095 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:44.266Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:43.943Z InvocationID=1039aa4a-0fb4-4713-86c4-d64c832cc095 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:44.267Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:43.943Z InvocationID=1039aa4a-0fb4-4713-86c4-d64c832cc095 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:44.267Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:43.943Z InvocationID=1039aa4a-0fb4-4713-86c4-d64c832cc095 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:44.349Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:43.943Z InvocationID=1039aa4a-0fb4-4713-86c4-d64c832cc095 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name VlanNetworkReceptorConfiguration 2021-08-11T16:12:45.803Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VLAN Network Receptor Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "46c5927f-c063-4a9c-8b99-28699adcf027" INVARIANT_UUID = "25e4f870-3b3f-4d34-b437-f3dc2cd9adf5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.nodes.VLANNetworkReceptor" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=46c5927f-c063-4a9c-8b99-28699adcf027 EntryTimestamp=2021-08-11T16:12:43.943Z InvocationID=1039aa4a-0fb4-4713-86c4-d64c832cc095 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id 5a023316-81c3-4451-96f2-24ab7cb55a51 2021-08-11T16:12:46.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0fb6136e-5d63-461a-941f-6edace8ce808 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:12:46.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0fb6136e-5d63-461a-941f-6edace8ce808 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2cbacc69 2021-08-11T16:12:46.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0fb6136e-5d63-461a-941f-6edace8ce808 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:12:46.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0fb6136e-5d63-461a-941f-6edace8ce808 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:12:47.402Z [qtp2045766957-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VLAN Network Receptor Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "46c5927f-c063-4a9c-8b99-28699adcf027" INVARIANT_UUID = "25e4f870-3b3f-4d34-b437-f3dc2cd9adf5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "5a023316-81c3-4451-96f2-24ab7cb55a51.assettoscatemplate" CURR_ARTIFACT_UUID = "5a023316-81c3-4451-96f2-24ab7cb55a51.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VlanNetworkReceptorConfiguration-template.yml,0,5a023316-81c3-4451-96f2-24ab7cb55a51.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=417fde8a-c77b-4763-a675-b4756de93ba9 EntryTimestamp=2021-08-11T16:12:43.943Z InvocationID=1039aa4a-0fb4-4713-86c4-d64c832cc095 PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Starting to create Csar for component VLAN Network Receptor Configuration 2021-08-11T16:12:47.513Z [qtp2045766957-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VLAN Network Receptor Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "46c5927f-c063-4a9c-8b99-28699adcf027" INVARIANT_UUID = "25e4f870-3b3f-4d34-b437-f3dc2cd9adf5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "5a023316-81c3-4451-96f2-24ab7cb55a51.assettoscatemplate" CURR_ARTIFACT_UUID = "5a023316-81c3-4451-96f2-24ab7cb55a51.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VlanNetworkReceptorConfiguration-template.yml,0,5a023316-81c3-4451-96f2-24ab7cb55a51.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=417fde8a-c77b-4763-a675-b4756de93ba9 EntryTimestamp=2021-08-11T16:12:43.943Z InvocationID=1039aa4a-0fb4-4713-86c4-d64c832cc095 Ended create Csar for component VLAN Network Receptor Configuration 2021-08-11T16:12:47.514Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VLAN Network Receptor Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "46c5927f-c063-4a9c-8b99-28699adcf027" INVARIANT_UUID = "25e4f870-3b3f-4d34-b437-f3dc2cd9adf5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "5a023316-81c3-4451-96f2-24ab7cb55a51.assettoscatemplate" CURR_ARTIFACT_UUID = "5a023316-81c3-4451-96f2-24ab7cb55a51.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VlanNetworkReceptorConfiguration-template.yml,0,5a023316-81c3-4451-96f2-24ab7cb55a51.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=417fde8a-c77b-4763-a675-b4756de93ba9 EntryTimestamp=2021-08-11T16:12:43.943Z InvocationID=1039aa4a-0fb4-4713-86c4-d64c832cc095 lock resource with id 5a023316-81c3-4451-96f2-24ab7cb55a51 2021-08-11T16:12:48.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:12:48.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:12:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:12:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:12:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:12:49.360Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VLAN Network Receptor Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "46c5927f-c063-4a9c-8b99-28699adcf027" INVARIANT_UUID = "25e4f870-3b3f-4d34-b437-f3dc2cd9adf5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "5a023316-81c3-4451-96f2-24ab7cb55a51.assettoscacsar" CURR_ARTIFACT_UUID = "5a023316-81c3-4451-96f2-24ab7cb55a51.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VlanNetworkReceptorConfiguration-csar.csar,0,5a023316-81c3-4451-96f2-24ab7cb55a51.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5417 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f7c2b212-1053-45c5-8e4d-f09e1f950b5e EntryTimestamp=2021-08-11T16:12:43.943Z InvocationID=1039aa4a-0fb4-4713-86c4-d64c832cc095 LogTimestamp=2021-08-11T16:12:49.360374Z ResponseDesc=Created Exiting. 2021-08-11T16:12:49.361Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:12:49.428Z [qtp2045766957-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=3d00707b-a7dc-4538-a47b-349d58d08e13 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:12:49.428342Z InvocationID=aab13ced-4046-4a90-b1ac-aad01a19d384 LogTimestamp=2021-08-11T16:12:49.428382Z Entering 2021-08-11T16:12:49.428Z [qtp2045766957-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=3d00707b-a7dc-4538-a47b-349d58d08e13 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:12:49.428342Z InvocationID=aab13ced-4046-4a90-b1ac-aad01a19d384 LogTimestamp=2021-08-11T16:12:49.428382Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:12:50.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5898bc3-edc7-405f-997e-5e1b1e0aab30 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:12:50.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5898bc3-edc7-405f-997e-5e1b1e0aab30 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4ffb5f92 2021-08-11T16:12:50.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5898bc3-edc7-405f-997e-5e1b1e0aab30 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:12:50.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5898bc3-edc7-405f-997e-5e1b1e0aab30 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:12:52.372Z [qtp2045766957-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:52.373Z [qtp2045766957-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:52.373Z [qtp2045766957-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:52.374Z [qtp2045766957-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:12:52.646Z [qtp2045766957-20] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name VrfEntryConfiguration 2021-08-11T16:12:53.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd2e0377-2b16-420a-a68a-047406850403 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:12:53.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd2e0377-2b16-420a-a68a-047406850403 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1547bb9e 2021-08-11T16:12:53.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd2e0377-2b16-420a-a68a-047406850403 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:12:53.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd2e0377-2b16-420a-a68a-047406850403 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:12:56.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=277cb190-8e0b-4444-9022-e2aea64c8b28 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:12:56.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=277cb190-8e0b-4444-9022-e2aea64c8b28 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@f37f284 2021-08-11T16:12:56.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=277cb190-8e0b-4444-9022-e2aea64c8b28 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:12:56.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=277cb190-8e0b-4444-9022-e2aea64c8b28 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:12:57.467Z [qtp2045766957-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.nodes.VRFEntry" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b18a4061-7814-40e6-adf0-5e5c58b0c08a EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id b63ef510-9e61-4641-a2a9-c026eba3b5da 2021-08-11T16:12:59.295Z [cluster1-nio-worker-0] WARN c.d.driver.core.HostConnectionPool Error creating connection to cassandra.onap/10.242.184.131:9042 com.datastax.driver.core.exceptions.TransportException: [cassandra.onap/10.242.184.131:9042] Connection has been closed at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1427) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1408) at com.datastax.driver.core.Connection.defunct(Connection.java:585) at com.datastax.driver.core.Connection$Dispatcher.exceptionCaught(Connection.java:1330) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273) at io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.handleReadException(AbstractEpollStreamChannel.java:728) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:818) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-08-11T16:12:59.295Z [cluster1-nio-worker-0] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.184.131:9042 com.datastax.driver.core.exceptions.ConnectionException: [cassandra.onap/10.242.184.131:9042] Pool was closed during initialization at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:173) at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:158) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1080) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.google.common.util.concurrent.CollectionFuture.handleAllCompleted(CollectionFuture.java:69) at com.google.common.util.concurrent.AggregateFuture.processCompleted(AggregateFuture.java:282) at com.google.common.util.concurrent.AggregateFuture.decrementCountAndMaybeComplete(AggregateFuture.java:264) at com.google.common.util.concurrent.AggregateFuture.access$200(AggregateFuture.java:42) at com.google.common.util.concurrent.AggregateFuture$1.run(AggregateFuture.java:147) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:100) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.datastax.driver.core.Connection$Future.onException(Connection.java:1518) at com.datastax.driver.core.Connection$Dispatcher.errorOutAllHandler(Connection.java:1340) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1427) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1408) at com.datastax.driver.core.Connection.defunct(Connection.java:585) at com.datastax.driver.core.Connection$Dispatcher.exceptionCaught(Connection.java:1330) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273) at io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.handleReadException(AbstractEpollStreamChannel.java:728) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:818) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-08-11T16:12:59.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=420943c5-0700-43f9-ad46-9fb687808148 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:12:59.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=420943c5-0700-43f9-ad46-9fb687808148 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7594ae8f 2021-08-11T16:12:59.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=420943c5-0700-43f9-ad46-9fb687808148 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:12:59.762Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=420943c5-0700-43f9-ad46-9fb687808148 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:13:00.103Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=420943c5-0700-43f9-ad46-9fb687808148 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:13:00.103Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=420943c5-0700-43f9-ad46-9fb687808148 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:13:00.281Z [qtp2045766957-20] WARN o.j.d.l.c.ConsistentKeyLocker AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Temporary storage exception while deleting lock org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.mutate(CQLKeyColumnValueStore.java:363) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.deleteSingleLock(ConsistentKeyLocker.java:538) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.deleteSingleLock(ConsistentKeyLocker.java:118) at org.janusgraph.diskstorage.locking.AbstractLocker.deleteLocks(AbstractLocker.java:381) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStore.deleteLocks(ExpectedValueCheckingStore.java:115) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.deleteAllLocks(ExpectedValueCheckingTransaction.java:282) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.commit(ExpectedValueCheckingTransaction.java:92) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:197) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:733) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1379) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:298) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.unlockNode(JanusGraphGenericDao.java:1272) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.releaseElement(JanusGraphGenericDao.java:1284) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.unlockComponent(GraphLockOperation.java:65) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.unlockComponent(BaseBusinessLogic.java:290) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:877) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$3(ComponentBusinessLogic.java:496) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:494) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:00.351Z [qtp2045766957-20] WARN o.j.d.l.c.ConsistentKeyLocker AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Temporary storage exception while deleting lock org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.mutate(CQLKeyColumnValueStore.java:363) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.deleteSingleLock(ConsistentKeyLocker.java:538) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.deleteSingleLock(ConsistentKeyLocker.java:118) at org.janusgraph.diskstorage.locking.AbstractLocker.deleteLocks(AbstractLocker.java:381) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStore.deleteLocks(ExpectedValueCheckingStore.java:115) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.deleteAllLocks(ExpectedValueCheckingTransaction.java:282) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.commit(ExpectedValueCheckingTransaction.java:92) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:197) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:733) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1379) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:298) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.unlockNode(JanusGraphGenericDao.java:1272) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.releaseElement(JanusGraphGenericDao.java:1284) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.unlockComponent(GraphLockOperation.java:65) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.unlockComponent(BaseBusinessLogic.java:290) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:877) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$3(ComponentBusinessLogic.java:496) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:494) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:00.355Z [qtp2045766957-20] WARN o.j.d.l.c.ConsistentKeyLocker AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Temporary storage exception while deleting lock org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.mutate(CQLKeyColumnValueStore.java:363) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.deleteSingleLock(ConsistentKeyLocker.java:538) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.deleteSingleLock(ConsistentKeyLocker.java:118) at org.janusgraph.diskstorage.locking.AbstractLocker.deleteLocks(AbstractLocker.java:381) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStore.deleteLocks(ExpectedValueCheckingStore.java:115) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.deleteAllLocks(ExpectedValueCheckingTransaction.java:282) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.commit(ExpectedValueCheckingTransaction.java:92) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:197) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:733) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1379) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:298) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.unlockNode(JanusGraphGenericDao.java:1272) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.releaseElement(JanusGraphGenericDao.java:1284) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.unlockComponent(GraphLockOperation.java:65) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.unlockComponent(BaseBusinessLogic.java:290) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:877) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$3(ComponentBusinessLogic.java:496) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:494) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:00.360Z [qtp2045766957-20] WARN o.j.d.l.c.ConsistentKeyLocker AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Temporary storage exception while deleting lock org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.mutate(CQLKeyColumnValueStore.java:363) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.deleteSingleLock(ConsistentKeyLocker.java:538) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.deleteSingleLock(ConsistentKeyLocker.java:118) at org.janusgraph.diskstorage.locking.AbstractLocker.deleteLocks(AbstractLocker.java:381) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStore.deleteLocks(ExpectedValueCheckingStore.java:115) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.deleteAllLocks(ExpectedValueCheckingTransaction.java:282) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.commit(ExpectedValueCheckingTransaction.java:92) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:197) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:733) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1379) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:298) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.unlockNode(JanusGraphGenericDao.java:1272) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.releaseElement(JanusGraphGenericDao.java:1284) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.unlockComponent(GraphLockOperation.java:65) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.unlockComponent(BaseBusinessLogic.java:290) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:877) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$3(ComponentBusinessLogic.java:496) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:494) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:00.453Z [qtp2045766957-20] WARN o.j.d.l.c.ConsistentKeyLocker AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Temporary storage exception while deleting lock org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.mutate(CQLKeyColumnValueStore.java:363) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.deleteSingleLock(ConsistentKeyLocker.java:538) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.deleteSingleLock(ConsistentKeyLocker.java:118) at org.janusgraph.diskstorage.locking.AbstractLocker.deleteLocks(AbstractLocker.java:381) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStore.deleteLocks(ExpectedValueCheckingStore.java:115) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.deleteAllLocks(ExpectedValueCheckingTransaction.java:282) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.commit(ExpectedValueCheckingTransaction.java:92) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:197) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:733) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1379) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:298) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.unlockNode(JanusGraphGenericDao.java:1272) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.releaseElement(JanusGraphGenericDao.java:1284) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.unlockComponent(GraphLockOperation.java:65) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.unlockComponent(BaseBusinessLogic.java:290) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:877) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$3(ComponentBusinessLogic.java:496) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:494) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:00.457Z [qtp2045766957-20] WARN o.j.d.l.c.ConsistentKeyLocker AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Temporary storage exception while deleting lock org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.mutate(CQLKeyColumnValueStore.java:363) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.deleteSingleLock(ConsistentKeyLocker.java:538) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.deleteSingleLock(ConsistentKeyLocker.java:118) at org.janusgraph.diskstorage.locking.AbstractLocker.deleteLocks(AbstractLocker.java:381) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStore.deleteLocks(ExpectedValueCheckingStore.java:115) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.deleteAllLocks(ExpectedValueCheckingTransaction.java:282) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.commit(ExpectedValueCheckingTransaction.java:92) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:197) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:733) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1379) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:298) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.unlockNode(JanusGraphGenericDao.java:1272) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.releaseElement(JanusGraphGenericDao.java:1284) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.unlockComponent(GraphLockOperation.java:65) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.unlockComponent(BaseBusinessLogic.java:290) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:877) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$3(ComponentBusinessLogic.java:496) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:494) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:00.460Z [qtp2045766957-20] WARN o.j.d.l.c.ConsistentKeyLocker AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Temporary storage exception while deleting lock org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.mutate(CQLKeyColumnValueStore.java:363) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.deleteSingleLock(ConsistentKeyLocker.java:538) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.deleteSingleLock(ConsistentKeyLocker.java:118) at org.janusgraph.diskstorage.locking.AbstractLocker.deleteLocks(AbstractLocker.java:381) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStore.deleteLocks(ExpectedValueCheckingStore.java:115) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.deleteAllLocks(ExpectedValueCheckingTransaction.java:282) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.commit(ExpectedValueCheckingTransaction.java:92) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:197) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:733) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1379) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:298) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.unlockNode(JanusGraphGenericDao.java:1272) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.releaseElement(JanusGraphGenericDao.java:1284) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.unlockComponent(GraphLockOperation.java:65) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.unlockComponent(BaseBusinessLogic.java:290) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:877) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$3(ComponentBusinessLogic.java:496) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:494) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:00.549Z [qtp2045766957-20] WARN o.j.d.l.c.ConsistentKeyLocker AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Temporary storage exception while deleting lock org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.mutate(CQLKeyColumnValueStore.java:363) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.deleteSingleLock(ConsistentKeyLocker.java:538) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.deleteSingleLock(ConsistentKeyLocker.java:118) at org.janusgraph.diskstorage.locking.AbstractLocker.deleteLocks(AbstractLocker.java:381) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStore.deleteLocks(ExpectedValueCheckingStore.java:115) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.deleteAllLocks(ExpectedValueCheckingTransaction.java:282) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.commit(ExpectedValueCheckingTransaction.java:92) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:197) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:733) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1379) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:298) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.unlockNode(JanusGraphGenericDao.java:1272) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.releaseElement(JanusGraphGenericDao.java:1284) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.unlockComponent(GraphLockOperation.java:65) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.unlockComponent(BaseBusinessLogic.java:290) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:877) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$3(ComponentBusinessLogic.java:496) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:494) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:00.553Z [qtp2045766957-20] WARN o.j.d.l.c.ConsistentKeyLocker AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Temporary storage exception while deleting lock org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.mutate(CQLKeyColumnValueStore.java:363) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.deleteSingleLock(ConsistentKeyLocker.java:538) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.deleteSingleLock(ConsistentKeyLocker.java:118) at org.janusgraph.diskstorage.locking.AbstractLocker.deleteLocks(AbstractLocker.java:381) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStore.deleteLocks(ExpectedValueCheckingStore.java:115) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.deleteAllLocks(ExpectedValueCheckingTransaction.java:282) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.commit(ExpectedValueCheckingTransaction.java:92) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:197) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:733) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1379) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:298) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.unlockNode(JanusGraphGenericDao.java:1272) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.releaseElement(JanusGraphGenericDao.java:1284) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.unlockComponent(GraphLockOperation.java:65) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.unlockComponent(BaseBusinessLogic.java:290) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:877) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$3(ComponentBusinessLogic.java:496) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:494) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:00.559Z [qtp2045766957-20] WARN o.j.d.l.c.ConsistentKeyLocker AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Temporary storage exception while deleting lock org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.mutate(CQLKeyColumnValueStore.java:363) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.deleteSingleLock(ConsistentKeyLocker.java:538) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.deleteSingleLock(ConsistentKeyLocker.java:118) at org.janusgraph.diskstorage.locking.AbstractLocker.deleteLocks(AbstractLocker.java:381) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStore.deleteLocks(ExpectedValueCheckingStore.java:115) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.deleteAllLocks(ExpectedValueCheckingTransaction.java:282) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.commit(ExpectedValueCheckingTransaction.java:92) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:197) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:733) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1379) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:298) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.unlockNode(JanusGraphGenericDao.java:1272) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.releaseElement(JanusGraphGenericDao.java:1284) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.unlockComponent(GraphLockOperation.java:65) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.unlockComponent(BaseBusinessLogic.java:290) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:877) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$3(ComponentBusinessLogic.java:496) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:494) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:00.652Z [qtp2045766957-20] WARN o.j.d.l.c.ConsistentKeyLocker AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Temporary storage exception while deleting lock org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.mutate(CQLKeyColumnValueStore.java:363) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.deleteSingleLock(ConsistentKeyLocker.java:538) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.deleteSingleLock(ConsistentKeyLocker.java:118) at org.janusgraph.diskstorage.locking.AbstractLocker.deleteLocks(AbstractLocker.java:381) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStore.deleteLocks(ExpectedValueCheckingStore.java:115) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.deleteAllLocks(ExpectedValueCheckingTransaction.java:282) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.commit(ExpectedValueCheckingTransaction.java:92) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:197) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:733) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1379) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:298) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.unlockNode(JanusGraphGenericDao.java:1272) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.releaseElement(JanusGraphGenericDao.java:1284) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.unlockComponent(GraphLockOperation.java:65) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.unlockComponent(BaseBusinessLogic.java:290) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:877) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$3(ComponentBusinessLogic.java:496) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:494) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:00.657Z [qtp2045766957-20] WARN o.j.d.l.c.ConsistentKeyLocker AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Temporary storage exception while deleting lock org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.mutate(CQLKeyColumnValueStore.java:363) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.deleteSingleLock(ConsistentKeyLocker.java:538) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.deleteSingleLock(ConsistentKeyLocker.java:118) at org.janusgraph.diskstorage.locking.AbstractLocker.deleteLocks(AbstractLocker.java:381) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStore.deleteLocks(ExpectedValueCheckingStore.java:115) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.deleteAllLocks(ExpectedValueCheckingTransaction.java:282) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.commit(ExpectedValueCheckingTransaction.java:92) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:197) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:733) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1379) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:298) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.unlockNode(JanusGraphGenericDao.java:1272) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.releaseElement(JanusGraphGenericDao.java:1284) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.unlockComponent(GraphLockOperation.java:65) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.unlockComponent(BaseBusinessLogic.java:290) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:877) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$3(ComponentBusinessLogic.java:496) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:494) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:00.750Z [qtp2045766957-20] WARN o.j.d.l.c.ConsistentKeyLocker AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Temporary storage exception while deleting lock org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.mutate(CQLKeyColumnValueStore.java:363) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.deleteSingleLock(ConsistentKeyLocker.java:538) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.deleteSingleLock(ConsistentKeyLocker.java:118) at org.janusgraph.diskstorage.locking.AbstractLocker.deleteLocks(AbstractLocker.java:381) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStore.deleteLocks(ExpectedValueCheckingStore.java:115) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.deleteAllLocks(ExpectedValueCheckingTransaction.java:282) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.commit(ExpectedValueCheckingTransaction.java:92) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:197) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:733) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1379) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:298) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.unlockNode(JanusGraphGenericDao.java:1272) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.releaseElement(JanusGraphGenericDao.java:1284) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.unlockComponent(GraphLockOperation.java:65) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.unlockComponent(BaseBusinessLogic.java:290) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:877) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$3(ComponentBusinessLogic.java:496) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:494) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:00.756Z [qtp2045766957-20] WARN o.j.d.l.c.ConsistentKeyLocker AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Temporary storage exception while deleting lock org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.mutate(CQLKeyColumnValueStore.java:363) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.deleteSingleLock(ConsistentKeyLocker.java:538) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.deleteSingleLock(ConsistentKeyLocker.java:118) at org.janusgraph.diskstorage.locking.AbstractLocker.deleteLocks(AbstractLocker.java:381) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStore.deleteLocks(ExpectedValueCheckingStore.java:115) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.deleteAllLocks(ExpectedValueCheckingTransaction.java:282) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.commit(ExpectedValueCheckingTransaction.java:92) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:197) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:733) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1379) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:298) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.unlockNode(JanusGraphGenericDao.java:1272) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.releaseElement(JanusGraphGenericDao.java:1284) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.unlockComponent(GraphLockOperation.java:65) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.unlockComponent(BaseBusinessLogic.java:290) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:877) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$3(ComponentBusinessLogic.java:496) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:494) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:00.760Z [qtp2045766957-20] WARN o.j.d.l.c.ConsistentKeyLocker AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Temporary storage exception while deleting lock org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateManyUnlogged(CQLStoreManager.java:504) at org.janusgraph.diskstorage.cql.CQLStoreManager.mutateMany(CQLStoreManager.java:435) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.mutate(CQLKeyColumnValueStore.java:363) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.deleteSingleLock(ConsistentKeyLocker.java:538) at org.janusgraph.diskstorage.locking.consistentkey.ConsistentKeyLocker.deleteSingleLock(ConsistentKeyLocker.java:118) at org.janusgraph.diskstorage.locking.AbstractLocker.deleteLocks(AbstractLocker.java:381) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingStore.deleteLocks(ExpectedValueCheckingStore.java:115) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.deleteAllLocks(ExpectedValueCheckingTransaction.java:282) at org.janusgraph.diskstorage.locking.consistentkey.ExpectedValueCheckingTransaction.commit(ExpectedValueCheckingTransaction.java:92) at org.janusgraph.diskstorage.keycolumnvalue.cache.CacheTransaction.commit(CacheTransaction.java:197) at org.janusgraph.diskstorage.BackendTransaction.commitStorage(BackendTransaction.java:134) at org.janusgraph.graphdb.database.StandardJanusGraph.commit(StandardJanusGraph.java:733) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.commit(StandardJanusGraphTx.java:1379) at org.janusgraph.graphdb.tinkerpop.JanusGraphBlueprintsGraph$GraphTransaction.doCommit(JanusGraphBlueprintsGraph.java:298) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:104) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient.commit(JanusGraphClient.java:250) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.unlockNode(JanusGraphGenericDao.java:1272) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.releaseElement(JanusGraphGenericDao.java:1284) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.unlockComponent(GraphLockOperation.java:65) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.unlockComponent(BaseBusinessLogic.java:290) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:877) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$3(ComponentBusinessLogic.java:496) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:494) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:00.760Z [qtp2045766957-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Starting to create Csar for component VRF Entry Configuration 2021-08-11T16:13:01.003Z [qtp2045766957-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 Ended create Csar for component VRF Entry Configuration 2021-08-11T16:13:01.003Z [qtp2045766957-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 lock resource with id b63ef510-9e61-4641-a2a9-c026eba3b5da 2021-08-11T16:13:01.052Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at com.google.common.collect.Iterators$5.computeNext(Iterators.java:635) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at com.google.common.collect.Iterables.isEmpty(Iterables.java:988) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.addProperty(StandardJanusGraphTx.java:816) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.addProperty(StandardJanusGraphTx.java:745) at org.janusgraph.graphdb.vertices.AbstractVertex.property(AbstractVertex.java:152) at org.janusgraph.core.JanusGraphVertex.property(JanusGraphVertex.java:72) at org.janusgraph.core.JanusGraphVertex.property(JanusGraphVertex.java:33) at org.apache.tinkerpop.gremlin.structure.util.ElementHelper.attachProperties(ElementHelper.java:285) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.setProperties(JanusGraphGenericDao.java:1485) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.createNode(JanusGraphGenericDao.java:117) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockNode(JanusGraphGenericDao.java:1127) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockElement(JanusGraphGenericDao.java:1117) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.lockComponent(GraphLockOperation.java:51) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockElement(BaseBusinessLogic.java:269) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:249) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:244) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:862) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:01.156Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at com.google.common.collect.Iterators$5.computeNext(Iterators.java:635) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at com.google.common.collect.Iterables.isEmpty(Iterables.java:988) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.addProperty(StandardJanusGraphTx.java:816) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.addProperty(StandardJanusGraphTx.java:745) at org.janusgraph.graphdb.vertices.AbstractVertex.property(AbstractVertex.java:152) at org.janusgraph.core.JanusGraphVertex.property(JanusGraphVertex.java:72) at org.janusgraph.core.JanusGraphVertex.property(JanusGraphVertex.java:33) at org.apache.tinkerpop.gremlin.structure.util.ElementHelper.attachProperties(ElementHelper.java:285) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.setProperties(JanusGraphGenericDao.java:1485) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.createNode(JanusGraphGenericDao.java:117) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockNode(JanusGraphGenericDao.java:1127) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockElement(JanusGraphGenericDao.java:1117) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.lockComponent(GraphLockOperation.java:51) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockElement(BaseBusinessLogic.java:269) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:249) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:244) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:862) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:01.260Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at com.google.common.collect.Iterators$5.computeNext(Iterators.java:635) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at com.google.common.collect.Iterables.isEmpty(Iterables.java:988) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.addProperty(StandardJanusGraphTx.java:816) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.addProperty(StandardJanusGraphTx.java:745) at org.janusgraph.graphdb.vertices.AbstractVertex.property(AbstractVertex.java:152) at org.janusgraph.core.JanusGraphVertex.property(JanusGraphVertex.java:72) at org.janusgraph.core.JanusGraphVertex.property(JanusGraphVertex.java:33) at org.apache.tinkerpop.gremlin.structure.util.ElementHelper.attachProperties(ElementHelper.java:285) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.setProperties(JanusGraphGenericDao.java:1485) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.createNode(JanusGraphGenericDao.java:117) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockNode(JanusGraphGenericDao.java:1127) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockElement(JanusGraphGenericDao.java:1117) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.lockComponent(GraphLockOperation.java:51) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockElement(BaseBusinessLogic.java:269) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:249) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:244) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:862) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:01.369Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at com.google.common.collect.Iterators$5.computeNext(Iterators.java:635) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at com.google.common.collect.Iterables.isEmpty(Iterables.java:988) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.addProperty(StandardJanusGraphTx.java:816) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.addProperty(StandardJanusGraphTx.java:745) at org.janusgraph.graphdb.vertices.AbstractVertex.property(AbstractVertex.java:152) at org.janusgraph.core.JanusGraphVertex.property(JanusGraphVertex.java:72) at org.janusgraph.core.JanusGraphVertex.property(JanusGraphVertex.java:33) at org.apache.tinkerpop.gremlin.structure.util.ElementHelper.attachProperties(ElementHelper.java:285) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.setProperties(JanusGraphGenericDao.java:1485) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.createNode(JanusGraphGenericDao.java:117) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockNode(JanusGraphGenericDao.java:1127) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockElement(JanusGraphGenericDao.java:1117) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.lockComponent(GraphLockOperation.java:51) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockElement(BaseBusinessLogic.java:269) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:249) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:244) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:862) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:01.576Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at com.google.common.collect.Iterators$5.computeNext(Iterators.java:635) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at com.google.common.collect.Iterables.isEmpty(Iterables.java:988) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.addProperty(StandardJanusGraphTx.java:816) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.addProperty(StandardJanusGraphTx.java:745) at org.janusgraph.graphdb.vertices.AbstractVertex.property(AbstractVertex.java:152) at org.janusgraph.core.JanusGraphVertex.property(JanusGraphVertex.java:72) at org.janusgraph.core.JanusGraphVertex.property(JanusGraphVertex.java:33) at org.apache.tinkerpop.gremlin.structure.util.ElementHelper.attachProperties(ElementHelper.java:285) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.setProperties(JanusGraphGenericDao.java:1485) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.createNode(JanusGraphGenericDao.java:117) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockNode(JanusGraphGenericDao.java:1127) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockElement(JanusGraphGenericDao.java:1117) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.lockComponent(GraphLockOperation.java:51) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockElement(BaseBusinessLogic.java:269) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:249) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:244) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:862) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:01.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=920ab279-6370-4acc-a55a-7a29e10db6ba PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:13:01.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=920ab279-6370-4acc-a55a-7a29e10db6ba PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@60866860 2021-08-11T16:13:01.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=920ab279-6370-4acc-a55a-7a29e10db6ba PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:13:01.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=920ab279-6370-4acc-a55a-7a29e10db6ba PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:13:01.599Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=920ab279-6370-4acc-a55a-7a29e10db6ba ErrorCategory=INFO ServiceName=BE Component CASSANDRA state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component CASSANDRA state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:13:01.600Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=920ab279-6370-4acc-a55a-7a29e10db6ba ErrorCategory=ERROR ServiceName=BE Component CASSANDRA state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-08-11T16:13:01.984Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at com.google.common.collect.Iterators$5.computeNext(Iterators.java:635) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at com.google.common.collect.Iterables.isEmpty(Iterables.java:988) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.addProperty(StandardJanusGraphTx.java:816) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.addProperty(StandardJanusGraphTx.java:745) at org.janusgraph.graphdb.vertices.AbstractVertex.property(AbstractVertex.java:152) at org.janusgraph.core.JanusGraphVertex.property(JanusGraphVertex.java:72) at org.janusgraph.core.JanusGraphVertex.property(JanusGraphVertex.java:33) at org.apache.tinkerpop.gremlin.structure.util.ElementHelper.attachProperties(ElementHelper.java:285) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.setProperties(JanusGraphGenericDao.java:1485) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.createNode(JanusGraphGenericDao.java:117) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockNode(JanusGraphGenericDao.java:1127) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockElement(JanusGraphGenericDao.java:1117) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.lockComponent(GraphLockOperation.java:51) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockElement(BaseBusinessLogic.java:269) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:249) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:244) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:862) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:02.249Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:02.303Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:02.411Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:02.515Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:02.720Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:02.790Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at com.google.common.collect.Iterators$5.computeNext(Iterators.java:635) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at com.google.common.collect.Iterables.isEmpty(Iterables.java:988) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.addProperty(StandardJanusGraphTx.java:816) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.addProperty(StandardJanusGraphTx.java:745) at org.janusgraph.graphdb.vertices.AbstractVertex.property(AbstractVertex.java:152) at org.janusgraph.core.JanusGraphVertex.property(JanusGraphVertex.java:72) at org.janusgraph.core.JanusGraphVertex.property(JanusGraphVertex.java:33) at org.apache.tinkerpop.gremlin.structure.util.ElementHelper.attachProperties(ElementHelper.java:285) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.setProperties(JanusGraphGenericDao.java:1485) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.createNode(JanusGraphGenericDao.java:117) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockNode(JanusGraphGenericDao.java:1127) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockElement(JanusGraphGenericDao.java:1117) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.lockComponent(GraphLockOperation.java:51) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockElement(BaseBusinessLogic.java:269) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:249) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:244) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:862) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:03.125Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:03.245Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:13:03.245Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-08-11T16:13:03.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:13:03.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:13:03.373Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:03.426Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:03.482Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:03.536Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:03.590Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:03.593Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at com.google.common.collect.Iterators$5.computeNext(Iterators.java:635) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at com.google.common.collect.Iterables.isEmpty(Iterables.java:988) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.addProperty(StandardJanusGraphTx.java:816) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.addProperty(StandardJanusGraphTx.java:745) at org.janusgraph.graphdb.vertices.AbstractVertex.property(AbstractVertex.java:152) at org.janusgraph.core.JanusGraphVertex.property(JanusGraphVertex.java:72) at org.janusgraph.core.JanusGraphVertex.property(JanusGraphVertex.java:33) at org.apache.tinkerpop.gremlin.structure.util.ElementHelper.attachProperties(ElementHelper.java:285) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.setProperties(JanusGraphGenericDao.java:1485) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.createNode(JanusGraphGenericDao.java:117) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockNode(JanusGraphGenericDao.java:1127) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockElement(JanusGraphGenericDao.java:1117) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.lockComponent(GraphLockOperation.java:51) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockElement(BaseBusinessLogic.java:269) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:249) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:244) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:862) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:03.693Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:03.797Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:03.930Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:04.000Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:04.211Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:04.264Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:13:04.401Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at com.google.common.collect.Iterators$5.computeNext(Iterators.java:635) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at com.google.common.collect.Iterables.isEmpty(Iterables.java:988) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.addProperty(StandardJanusGraphTx.java:816) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.addProperty(StandardJanusGraphTx.java:745) at org.janusgraph.graphdb.vertices.AbstractVertex.property(AbstractVertex.java:152) at org.janusgraph.core.JanusGraphVertex.property(JanusGraphVertex.java:72) at org.janusgraph.core.JanusGraphVertex.property(JanusGraphVertex.java:33) at org.apache.tinkerpop.gremlin.structure.util.ElementHelper.attachProperties(ElementHelper.java:285) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.setProperties(JanusGraphGenericDao.java:1485) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.createNode(JanusGraphGenericDao.java:117) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockNode(JanusGraphGenericDao.java:1127) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockElement(JanusGraphGenericDao.java:1117) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.lockComponent(GraphLockOperation.java:51) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockElement(BaseBusinessLogic.java:269) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:249) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:244) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:862) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:04.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2dffed4e-c14e-4495-a63d-dd44c1d1db80 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:13:04.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2dffed4e-c14e-4495-a63d-dd44c1d1db80 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4835b4a5 2021-08-11T16:13:04.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2dffed4e-c14e-4495-a63d-dd44c1d1db80 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:13:04.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2dffed4e-c14e-4495-a63d-dd44c1d1db80 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:13:04.605Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=2dffed4e-c14e-4495-a63d-dd44c1d1db80 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:13:04.605Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=2dffed4e-c14e-4495-a63d-dd44c1d1db80 ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-08-11T16:13:04.614Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:05.417Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:05.533Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:06.006Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at com.google.common.collect.Iterators$5.computeNext(Iterators.java:635) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at com.google.common.collect.Iterables.isEmpty(Iterables.java:988) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.addProperty(StandardJanusGraphTx.java:816) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.addProperty(StandardJanusGraphTx.java:745) at org.janusgraph.graphdb.vertices.AbstractVertex.property(AbstractVertex.java:152) at org.janusgraph.core.JanusGraphVertex.property(JanusGraphVertex.java:72) at org.janusgraph.core.JanusGraphVertex.property(JanusGraphVertex.java:33) at org.apache.tinkerpop.gremlin.structure.util.ElementHelper.attachProperties(ElementHelper.java:285) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.setProperties(JanusGraphGenericDao.java:1485) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.createNode(JanusGraphGenericDao.java:117) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockNode(JanusGraphGenericDao.java:1127) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockElement(JanusGraphGenericDao.java:1117) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.lockComponent(GraphLockOperation.java:51) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockElement(BaseBusinessLogic.java:269) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:249) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:244) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:862) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:06.221Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:06.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:13:07.024Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T16:12:57.862205Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:07.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1af317ad-bd20-4d57-b510-a6384f546573 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:13:07.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1af317ad-bd20-4d57-b510-a6384f546573 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7826ccf5 2021-08-11T16:13:07.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1af317ad-bd20-4d57-b510-a6384f546573 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:13:07.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1af317ad-bd20-4d57-b510-a6384f546573 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:13:07.612Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at com.google.common.collect.Iterators$5.computeNext(Iterators.java:635) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at com.google.common.collect.Iterables.isEmpty(Iterables.java:988) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.addProperty(StandardJanusGraphTx.java:816) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx.addProperty(StandardJanusGraphTx.java:745) at org.janusgraph.graphdb.vertices.AbstractVertex.property(AbstractVertex.java:152) at org.janusgraph.core.JanusGraphVertex.property(JanusGraphVertex.java:72) at org.janusgraph.core.JanusGraphVertex.property(JanusGraphVertex.java:33) at org.apache.tinkerpop.gremlin.structure.util.ElementHelper.attachProperties(ElementHelper.java:285) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.setProperties(JanusGraphGenericDao.java:1485) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.createNode(JanusGraphGenericDao.java:117) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockNode(JanusGraphGenericDao.java:1127) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockElement(JanusGraphGenericDao.java:1117) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.lockComponent(GraphLockOperation.java:51) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockElement(BaseBusinessLogic.java:269) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:249) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:244) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:862) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:527) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:08.737Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:09.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:13:10.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87127aa6-91b9-4a98-ab20-c182b76f7e0e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:13:10.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87127aa6-91b9-4a98-ab20-c182b76f7e0e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2816e5a8 2021-08-11T16:13:10.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87127aa6-91b9-4a98-ab20-c182b76f7e0e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:13:10.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87127aa6-91b9-4a98-ab20-c182b76f7e0e PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:13:10.823Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at com.google.common.collect.Iterators$5.computeNext(Iterators.java:635) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockNode(JanusGraphGenericDao.java:1135) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockElement(JanusGraphGenericDao.java:1117) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.lockComponent(GraphLockOperation.java:51) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockElement(BaseBusinessLogic.java:269) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:249) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:244) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:862) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:10.851Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at com.google.common.collect.Iterators$5.computeNext(Iterators.java:635) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockNode(JanusGraphGenericDao.java:1135) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockElement(JanusGraphGenericDao.java:1117) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.lockComponent(GraphLockOperation.java:51) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockElement(BaseBusinessLogic.java:269) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:249) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:244) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:862) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:10.904Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at com.google.common.collect.Iterators$5.computeNext(Iterators.java:635) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockNode(JanusGraphGenericDao.java:1135) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockElement(JanusGraphGenericDao.java:1117) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.lockComponent(GraphLockOperation.java:51) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockElement(BaseBusinessLogic.java:269) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:249) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:244) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:862) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:10.957Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at com.google.common.collect.Iterators$5.computeNext(Iterators.java:635) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockNode(JanusGraphGenericDao.java:1135) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockElement(JanusGraphGenericDao.java:1117) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.lockComponent(GraphLockOperation.java:51) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockElement(BaseBusinessLogic.java:269) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:249) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:244) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:862) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:11.010Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at com.google.common.collect.Iterators$5.computeNext(Iterators.java:635) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockNode(JanusGraphGenericDao.java:1135) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockElement(JanusGraphGenericDao.java:1117) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.lockComponent(GraphLockOperation.java:51) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockElement(BaseBusinessLogic.java:269) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:249) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:244) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:862) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:11.114Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at com.google.common.collect.Iterators$5.computeNext(Iterators.java:635) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockNode(JanusGraphGenericDao.java:1135) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockElement(JanusGraphGenericDao.java:1117) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.lockComponent(GraphLockOperation.java:51) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockElement(BaseBusinessLogic.java:269) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:249) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:244) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:862) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:11.217Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at com.google.common.collect.Iterators$5.computeNext(Iterators.java:635) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockNode(JanusGraphGenericDao.java:1135) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockElement(JanusGraphGenericDao.java:1117) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.lockComponent(GraphLockOperation.java:51) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockElement(BaseBusinessLogic.java:269) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:249) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:244) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:862) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:11.420Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at com.google.common.collect.Iterators$5.computeNext(Iterators.java:635) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockNode(JanusGraphGenericDao.java:1135) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockElement(JanusGraphGenericDao.java:1117) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.lockComponent(GraphLockOperation.java:51) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockElement(BaseBusinessLogic.java:269) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:249) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:244) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:862) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:11.623Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at com.google.common.collect.Iterators$5.computeNext(Iterators.java:635) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockNode(JanusGraphGenericDao.java:1135) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockElement(JanusGraphGenericDao.java:1117) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.lockComponent(GraphLockOperation.java:51) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockElement(BaseBusinessLogic.java:269) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:249) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:244) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:862) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:11.829Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at com.google.common.collect.Iterators$5.computeNext(Iterators.java:635) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockNode(JanusGraphGenericDao.java:1135) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockElement(JanusGraphGenericDao.java:1117) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.lockComponent(GraphLockOperation.java:51) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockElement(BaseBusinessLogic.java:269) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:249) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:244) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:862) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:11.942Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:11.995Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:12.028Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:12.031Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at com.google.common.collect.Iterators$5.computeNext(Iterators.java:635) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockNode(JanusGraphGenericDao.java:1135) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockElement(JanusGraphGenericDao.java:1117) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.lockComponent(GraphLockOperation.java:51) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockElement(BaseBusinessLogic.java:269) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:249) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:244) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:862) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:12.048Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:12.055Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:12.085Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:12.100Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:12.112Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:12.153Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:12.164Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:12.235Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at com.google.common.collect.Iterators$5.computeNext(Iterators.java:635) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockNode(JanusGraphGenericDao.java:1135) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockElement(JanusGraphGenericDao.java:1117) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.lockComponent(GraphLockOperation.java:51) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockElement(BaseBusinessLogic.java:269) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:249) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:244) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:862) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:12.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:13:12.256Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:12.267Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:12.370Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:12.438Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at com.google.common.collect.Iterators$5.computeNext(Iterators.java:635) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockNode(JanusGraphGenericDao.java:1135) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockElement(JanusGraphGenericDao.java:1117) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.lockComponent(GraphLockOperation.java:51) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockElement(BaseBusinessLogic.java:269) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:249) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:244) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:862) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:12.459Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:12.580Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:12.642Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at com.google.common.collect.Iterators$5.computeNext(Iterators.java:635) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockNode(JanusGraphGenericDao.java:1135) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockElement(JanusGraphGenericDao.java:1117) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.lockComponent(GraphLockOperation.java:51) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockElement(BaseBusinessLogic.java:269) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:249) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:244) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:862) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:12.783Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:12.861Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:12.997Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:13.045Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at com.google.common.collect.Iterators$5.computeNext(Iterators.java:635) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockNode(JanusGraphGenericDao.java:1135) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockElement(JanusGraphGenericDao.java:1117) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.lockComponent(GraphLockOperation.java:51) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockElement(BaseBusinessLogic.java:269) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:249) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:244) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:862) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:13.264Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:13.409Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:13.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5df37f7-7c09-41b6-a457-d43df28f551b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:13:13.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5df37f7-7c09-41b6-a457-d43df28f551b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7b804da2 2021-08-11T16:13:13.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5df37f7-7c09-41b6-a457-d43df28f551b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:13:13.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5df37f7-7c09-41b6-a457-d43df28f551b PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:13:13.667Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:13.812Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:13.848Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at com.google.common.collect.Iterators$5.computeNext(Iterators.java:635) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockNode(JanusGraphGenericDao.java:1135) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockElement(JanusGraphGenericDao.java:1117) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.lockComponent(GraphLockOperation.java:51) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockElement(BaseBusinessLogic.java:269) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:249) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:244) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:862) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:14.215Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:14.470Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:15.018Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:15.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:13:15.452Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at com.google.common.collect.Iterators$5.computeNext(Iterators.java:635) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockNode(JanusGraphGenericDao.java:1135) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockElement(JanusGraphGenericDao.java:1117) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.lockComponent(GraphLockOperation.java:51) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockElement(BaseBusinessLogic.java:269) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:249) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:244) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:862) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:15.822Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T16:12:57.862205Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:16.073Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:16.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12d24c8c-ff40-4f84-9eeb-4f38b734bb1d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:13:16.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12d24c8c-ff40-4f84-9eeb-4f38b734bb1d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1ef54eeb 2021-08-11T16:13:16.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12d24c8c-ff40-4f84-9eeb-4f38b734bb1d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:13:16.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12d24c8c-ff40-4f84-9eeb-4f38b734bb1d PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:13:17.056Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at com.google.common.collect.Iterators$5.computeNext(Iterators.java:635) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockNode(JanusGraphGenericDao.java:1135) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.lockElement(JanusGraphGenericDao.java:1117) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.lockComponent(GraphLockOperation.java:51) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockElement(BaseBusinessLogic.java:269) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:249) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.lockComponent(BaseBusinessLogic.java:244) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:862) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:17.677Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:18.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:13:18.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:13:18.318Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:13:18.660Z [qtp2045766957-20] WARN o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=Artifact action - lock ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 TargetServiceName=null EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 ErrorCode=500 Failed to lock object for update. Type = resource, Id = b63ef510-9e61-4641-a2a9-c026eba3b5da 2021-08-11T16:13:18.663Z [qtp2045766957-20] WARN o.o.s.e.AbstractSdncException AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 TargetServiceName=null EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 ErrorCode=900 2021-08-11T16:13:18.664Z [qtp2045766957-20] WARN o.o.s.e.AbstractSdncException AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VrfEntryConfiguration-template.yml,0,b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 TargetServiceName=null EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 ErrorCode=900 2021-08-11T16:13:18.678Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscacsar" ARTIFACT_DATA = "" STATUS = "500" DESC = "POL5000: Error: Internal Server Error. Please try again later." SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 TargetServiceName=null EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 ErrorCode=900 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.deleteNode(JanusGraphGenericDao.java:917) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.deleteNode(JanusGraphGenericDao.java:907) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.unlockNode(JanusGraphGenericDao.java:1269) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.releaseElement(JanusGraphGenericDao.java:1284) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.unlockComponent(GraphLockOperation.java:65) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.unlockComponent(BaseBusinessLogic.java:290) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:877) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:18.731Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscacsar" ARTIFACT_DATA = "" STATUS = "500" DESC = "POL5000: Error: Internal Server Error. Please try again later." SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 TargetServiceName=null EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 ErrorCode=900 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.deleteNode(JanusGraphGenericDao.java:917) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.deleteNode(JanusGraphGenericDao.java:907) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.unlockNode(JanusGraphGenericDao.java:1269) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.releaseElement(JanusGraphGenericDao.java:1284) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.unlockComponent(GraphLockOperation.java:65) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.unlockComponent(BaseBusinessLogic.java:290) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:877) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:18.784Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscacsar" ARTIFACT_DATA = "" STATUS = "500" DESC = "POL5000: Error: Internal Server Error. Please try again later." SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 TargetServiceName=null EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 ErrorCode=900 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.deleteNode(JanusGraphGenericDao.java:917) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.deleteNode(JanusGraphGenericDao.java:907) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.unlockNode(JanusGraphGenericDao.java:1269) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.releaseElement(JanusGraphGenericDao.java:1284) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.unlockComponent(GraphLockOperation.java:65) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.unlockComponent(BaseBusinessLogic.java:290) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:877) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:18.837Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscacsar" ARTIFACT_DATA = "" STATUS = "500" DESC = "POL5000: Error: Internal Server Error. Please try again later." SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 TargetServiceName=null EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 ErrorCode=900 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.deleteNode(JanusGraphGenericDao.java:917) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.deleteNode(JanusGraphGenericDao.java:907) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.unlockNode(JanusGraphGenericDao.java:1269) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.releaseElement(JanusGraphGenericDao.java:1284) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.unlockComponent(GraphLockOperation.java:65) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.unlockComponent(BaseBusinessLogic.java:290) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:877) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:18.940Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscacsar" ARTIFACT_DATA = "" STATUS = "500" DESC = "POL5000: Error: Internal Server Error. Please try again later." SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 TargetServiceName=null EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 ErrorCode=900 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.deleteNode(JanusGraphGenericDao.java:917) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.deleteNode(JanusGraphGenericDao.java:907) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.unlockNode(JanusGraphGenericDao.java:1269) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.releaseElement(JanusGraphGenericDao.java:1284) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.unlockComponent(GraphLockOperation.java:65) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.unlockComponent(BaseBusinessLogic.java:290) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:877) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:19.146Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscacsar" ARTIFACT_DATA = "" STATUS = "500" DESC = "POL5000: Error: Internal Server Error. Please try again later." SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 TargetServiceName=null EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 ErrorCode=900 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.deleteNode(JanusGraphGenericDao.java:917) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.deleteNode(JanusGraphGenericDao.java:907) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.unlockNode(JanusGraphGenericDao.java:1269) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.releaseElement(JanusGraphGenericDao.java:1284) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.unlockComponent(GraphLockOperation.java:65) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.unlockComponent(BaseBusinessLogic.java:290) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:877) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:19.549Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscacsar" ARTIFACT_DATA = "" STATUS = "500" DESC = "POL5000: Error: Internal Server Error. Please try again later." SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 TargetServiceName=null EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 ErrorCode=900 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.deleteNode(JanusGraphGenericDao.java:917) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.deleteNode(JanusGraphGenericDao.java:907) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.unlockNode(JanusGraphGenericDao.java:1269) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.releaseElement(JanusGraphGenericDao.java:1284) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.unlockComponent(GraphLockOperation.java:65) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.unlockComponent(BaseBusinessLogic.java:290) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:877) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:19.576Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=819c450b-503b-4165-9d06-b0b19785e5ec PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:13:19.576Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=819c450b-503b-4165-9d06-b0b19785e5ec PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@507a78a4 2021-08-11T16:13:19.576Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=819c450b-503b-4165-9d06-b0b19785e5ec PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:13:19.576Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=819c450b-503b-4165-9d06-b0b19785e5ec PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:13:19.958Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscacsar" ARTIFACT_DATA = "" STATUS = "500" DESC = "POL5000: Error: Internal Server Error. Please try again later." SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 TargetServiceName=null EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 ErrorCode=900 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.deleteNode(JanusGraphGenericDao.java:917) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.deleteNode(JanusGraphGenericDao.java:907) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.unlockNode(JanusGraphGenericDao.java:1269) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.releaseElement(JanusGraphGenericDao.java:1284) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.unlockComponent(GraphLockOperation.java:65) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.unlockComponent(BaseBusinessLogic.java:290) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:877) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:20.762Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscacsar" ARTIFACT_DATA = "" STATUS = "500" DESC = "POL5000: Error: Internal Server Error. Please try again later." SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 TargetServiceName=null EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 ErrorCode=900 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.deleteNode(JanusGraphGenericDao.java:917) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.deleteNode(JanusGraphGenericDao.java:907) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.unlockNode(JanusGraphGenericDao.java:1269) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.releaseElement(JanusGraphGenericDao.java:1284) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.unlockComponent(GraphLockOperation.java:65) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.unlockComponent(BaseBusinessLogic.java:290) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:877) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:20.825Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:20.878Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:20.881Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:20.909Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:20.930Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:20.936Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:20.964Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:20.983Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:20.991Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:21.020Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:21.073Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:21.085Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:21.176Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:21.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:13:21.280Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:21.287Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:21.383Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:21.490Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:21.586Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:21.693Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:21.790Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:21.994Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:22.097Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:22.367Z [qtp2045766957-20] INFO o.j.d.util.BackendOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscacsar" ARTIFACT_DATA = "" STATUS = "500" DESC = "POL5000: Error: Internal Server Error. Please try again later." SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 TargetServiceName=null EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 ErrorCode=900 Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.deleteNode(JanusGraphGenericDao.java:917) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.deleteNode(JanusGraphGenericDao.java:907) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.unlockNode(JanusGraphGenericDao.java:1269) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.releaseElement(JanusGraphGenericDao.java:1284) at org.openecomp.sdc.be.model.operations.impl.GraphLockOperation.unlockComponent(GraphLockOperation.java:65) at org.openecomp.sdc.be.components.impl.BaseBusinessLogic.unlockComponent(BaseBusinessLogic.java:290) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lockComponentAndUpdateArtifact(ArtifactsBusinessLogic.java:877) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.lambda$generateAndSaveToscaArtifact$3(ArtifactsBusinessLogic.java:355) at fj.data.Either$LeftProjection.bind(Either.java:287) at org.openecomp.sdc.be.components.impl.ArtifactsBusinessLogic.generateAndSaveToscaArtifact(ArtifactsBusinessLogic.java:350) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactPayload(ComponentBusinessLogic.java:515) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.lambda$populateToscaArtifacts$4(ComponentBusinessLogic.java:500) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.saveToscaArtifactAndPopulateToscaArtifactsWithResult(ComponentBusinessLogic.java:145) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:499) at org.openecomp.sdc.be.components.impl.ComponentBusinessLogic.populateToscaArtifacts(ComponentBusinessLogic.java:480) at org.openecomp.sdc.be.components.impl.ResourceBusinessLogic.propagateStateToCertified(ResourceBusinessLogic.java:3258) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importCertifiedResource(ResourceImportManager.java:200) at org.openecomp.sdc.be.components.impl.ResourceImportManager.importNormativeResource(ResourceImportManager.java:143) at org.openecomp.sdc.be.servlets.AbstractValidationsServlet.handleImport(AbstractValidationsServlet.java:659) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart_aroundBody0(ResourceUploadServlet.java:131) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$AjcClosure1.run(ResourceUploadServlet.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at org.openecomp.sdc.be.servlets.ResourceUploadServlet.uploadMultipart(ResourceUploadServlet.java:109) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$FastClassBySpringCGLIB$$6732af97.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.adapter.MethodBeforeAdviceInterceptor.invoke(MethodBeforeAdviceInterceptor.java:56) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:95) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:749) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:691) at org.openecomp.sdc.be.servlets.ResourceUploadServlet$$EnhancerBySpringCGLIB$$c4546c90.uploadMultipart() at jdk.internal.reflect.GeneratedMethodAccessor359.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191) at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200) at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103) at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415) at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104) at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268) at org.glassfish.jersey.internal.Errors.process(Errors.java:316) at org.glassfish.jersey.internal.Errors.process(Errors.java:298) at org.glassfish.jersey.internal.Errors.process(Errors.java:268) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229) at org.eclipse.jetty.servlet.ServletHolder$NotAsyncServlet.service(ServletHolder.java:1411) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:763) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1651) at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:226) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.togglz.servlet.TogglzFilter.doFilter(TogglzFilter.java:100) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.ReqValidationFilter.doFilter(ReqValidationFilter.java:72) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1638) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:71) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:22.396Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:22.500Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:22.587Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf2911b5-4020-401d-ae7d-2386c1bd1678 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:13:22.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf2911b5-4020-401d-ae7d-2386c1bd1678 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4fdeeca6 2021-08-11T16:13:22.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf2911b5-4020-401d-ae7d-2386c1bd1678 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:13:22.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf2911b5-4020-401d-ae7d-2386c1bd1678 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:13:22.800Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:22.903Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:23.605Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:527) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:23.706Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T16:12:57.862205Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:24.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:13:24.408Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:25.571Z [qtp2045766957-20] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscacsar" ARTIFACT_DATA = "" STATUS = "500" DESC = "POL5000: Error: Internal Server Error. Please try again later." SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=Change LifecycleState - Certify ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 TargetServiceName=null EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 ErrorCode=500 Unexpected error during operation 2021-08-11T16:13:25.571Z [qtp2045766957-20] WARN o.o.s.e.AbstractSdncException AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscacsar" ARTIFACT_DATA = "" STATUS = "500" DESC = "POL5000: Error: Internal Server Error. Please try again later." SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 TargetServiceName=null EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 ErrorCode=900 2021-08-11T16:13:25.574Z [qtp2045766957-20] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "b18a4061-7814-40e6-adf0-5e5c58b0c08a" INVARIANT_UUID = "de43cdda-6173-49fe-b723-0e2181d7429a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "" CURR_ARTIFACT_UUID = "b63ef510-9e61-4641-a2a9-c026eba3b5da.assettoscacsar" ARTIFACT_DATA = "" STATUS = "500" DESC = "POL5000: Error: Internal Server Error. Please try again later." SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=Import Resource VRFEntry.yml ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 TargetServiceName=null EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 ErrorCode=500 Unexpected error during operation 2021-08-11T16:13:25.583Z [qtp2045766957-20] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "" INVARIANT_UUID = "" PREV_VERSION = "1.0" CURR_VERSION = "" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "" STATUS = "500" DESC = "POL5000: Error: Internal Server Error. Please try again later." TOSCA_NODE_TYPE = "" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=Upload Resource ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 TargetServiceName=null EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 ErrorCode=900 Unexpected error during BE REST API execution 2021-08-11T16:13:25.585Z [qtp2045766957-20] ERROR o.o.s.b.s.ResourceUploadServlet AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "" INVARIANT_UUID = "" PREV_VERSION = "1.0" CURR_VERSION = "" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "" STATUS = "500" DESC = "POL5000: Error: Internal Server Error. Please try again later." TOSCA_NODE_TYPE = "" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=Upload Resource ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 TargetServiceName=null EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 ErrorCode=900 #uploadMultipart('multipart', /tmp/rep18312318447906781325tmp, 'form-data; filename="VRFEntry.zip"; name="resourceZip"', '{"payloadName": "VRFEntry.yml", "contactId": "jh0003", "name": "VRF Entry Configuration", "description": "VRF Entry configuration object", "resourceIconPath": "pmc", "resourceType": "Configuration", "categories": [{"name": "Configuration", "subcategories": [{"name": "Configuration"}]}], "tags": ["VRF Entry Configuration"]}', 'Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/catalog/upload/multipart?createNewVersion=true)@52a4e9a9', 'jh0003', true): thrown org.openecomp.sdc.be.components.impl.exceptions.ByActionStatusComponentException out of org.openecomp.sdc.be.components.impl.BaseBusinessLogic#logAndThrowComponentException[263] in 36s 2021-08-11T16:13:25.587Z [qtp2045766957-20] WARN o.o.s.e.AbstractSdncException AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "" INVARIANT_UUID = "" PREV_VERSION = "1.0" CURR_VERSION = "" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "" STATUS = "500" DESC = "POL5000: Error: Internal Server Error. Please try again later." TOSCA_NODE_TYPE = "" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 TargetServiceName=null EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 ErrorCode=900 2021-08-11T16:13:25.587Z [qtp2045766957-20] WARN o.o.s.e.AbstractSdncException AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "" INVARIANT_UUID = "" PREV_VERSION = "1.0" CURR_VERSION = "" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "" STATUS = "500" DESC = "POL5000: Error: Internal Server Error. Please try again later." TOSCA_NODE_TYPE = "" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 TargetServiceName=null EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 ErrorCode=900 2021-08-11T16:13:25.589Z [qtp2045766957-20] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "" INVARIANT_UUID = "" PREV_VERSION = "1.0" CURR_VERSION = "" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "" STATUS = "500" DESC = "POL5000: Error: Internal Server Error. Please try again later." TOSCA_NODE_TYPE = "" SupportablityAction=GENERATE CSAR ResponseCode=500 RequestID=null ElapsedTime=36160 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=ERROR TargetEntity= ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ecd7e590-f55d-460f-a5a9-9bf9446d24c6 TargetServiceName=null ErrorDesc=Internal Server Error EntryTimestamp=2021-08-11T16:12:49.428Z InvocationID=24c6cbf3-da4e-431d-963d-bee7b0706404 ErrorCode=500 LogTimestamp=2021-08-11T16:13:25.588445Z ResponseDesc=Internal Server Error Exiting. 2021-08-11T16:13:25.590Z [qtp2045766957-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="500" 2021-08-11T16:13:25.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=431bdac5-9c59-4269-abea-025d321000a9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:13:25.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=431bdac5-9c59-4269-abea-025d321000a9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@364a45a2 2021-08-11T16:13:25.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=431bdac5-9c59-4269-abea-025d321000a9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:13:25.758Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=431bdac5-9c59-4269-abea-025d321000a9 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:13:26.011Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:27.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:13:27.648Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:28.748Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:28.847Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:28.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b757b5dd-1a01-4ecc-9718-6c140f71e5ff PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:13:28.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b757b5dd-1a01-4ecc-9718-6c140f71e5ff PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2fcc796 2021-08-11T16:13:28.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b757b5dd-1a01-4ecc-9718-6c140f71e5ff PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:13:28.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b757b5dd-1a01-4ecc-9718-6c140f71e5ff PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:13:28.957Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:527) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:29.059Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:29.162Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:29.251Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:29.364Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:29.768Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:30.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:13:30.572Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:30.855Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:30.883Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:30.935Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:31.038Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:31.240Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:31.604Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64acefd9-6563-4e79-9054-bc11f4390691 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:13:31.604Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64acefd9-6563-4e79-9054-bc11f4390691 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1c03035 2021-08-11T16:13:31.604Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64acefd9-6563-4e79-9054-bc11f4390691 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:13:31.604Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64acefd9-6563-4e79-9054-bc11f4390691 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:13:31.643Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:32.046Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:32.175Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T16:12:57.862205Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:32.450Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:32.852Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:33.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:13:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:13:33.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:13:33.655Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:34.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a82b380-2ae5-4c89-a4c8-8b855de9d786 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:13:34.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a82b380-2ae5-4c89-a4c8-8b855de9d786 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@39435777 2021-08-11T16:13:34.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a82b380-2ae5-4c89-a4c8-8b855de9d786 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:13:34.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a82b380-2ae5-4c89-a4c8-8b855de9d786 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:13:35.258Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:35.613Z [qtp2045766957-14] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getNode(JanusGraphGenericDao.java:317) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.validateUserExists(UserAdminOperation.java:129) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.getUserData(UserAdminOperation.java:77) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.getUserData(UserAdminOperation.java:69) at org.openecomp.sdc.be.user.UserBusinessLogic.getUser(UserBusinessLogic.java:71) at org.openecomp.sdc.be.filters.ThreadLocalUtils.updateUserContext(ThreadLocalUtils.java:84) at org.openecomp.sdc.be.filters.ThreadLocalUtils.setUserContextFromDB(ThreadLocalUtils.java:77) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:70) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:35.665Z [qtp2045766957-14] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getNode(JanusGraphGenericDao.java:317) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.validateUserExists(UserAdminOperation.java:129) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.getUserData(UserAdminOperation.java:77) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.getUserData(UserAdminOperation.java:69) at org.openecomp.sdc.be.user.UserBusinessLogic.getUser(UserBusinessLogic.java:71) at org.openecomp.sdc.be.filters.ThreadLocalUtils.updateUserContext(ThreadLocalUtils.java:84) at org.openecomp.sdc.be.filters.ThreadLocalUtils.setUserContextFromDB(ThreadLocalUtils.java:77) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:70) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:35.718Z [qtp2045766957-14] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getNode(JanusGraphGenericDao.java:317) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.validateUserExists(UserAdminOperation.java:129) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.getUserData(UserAdminOperation.java:77) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.getUserData(UserAdminOperation.java:69) at org.openecomp.sdc.be.user.UserBusinessLogic.getUser(UserBusinessLogic.java:71) at org.openecomp.sdc.be.filters.ThreadLocalUtils.updateUserContext(ThreadLocalUtils.java:84) at org.openecomp.sdc.be.filters.ThreadLocalUtils.setUserContextFromDB(ThreadLocalUtils.java:77) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:70) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:35.821Z [qtp2045766957-14] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getNode(JanusGraphGenericDao.java:317) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.validateUserExists(UserAdminOperation.java:129) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.getUserData(UserAdminOperation.java:77) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.getUserData(UserAdminOperation.java:69) at org.openecomp.sdc.be.user.UserBusinessLogic.getUser(UserBusinessLogic.java:71) at org.openecomp.sdc.be.filters.ThreadLocalUtils.updateUserContext(ThreadLocalUtils.java:84) at org.openecomp.sdc.be.filters.ThreadLocalUtils.setUserContextFromDB(ThreadLocalUtils.java:77) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:70) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:36.024Z [qtp2045766957-14] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getNode(JanusGraphGenericDao.java:317) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.validateUserExists(UserAdminOperation.java:129) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.getUserData(UserAdminOperation.java:77) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.getUserData(UserAdminOperation.java:69) at org.openecomp.sdc.be.user.UserBusinessLogic.getUser(UserBusinessLogic.java:71) at org.openecomp.sdc.be.filters.ThreadLocalUtils.updateUserContext(ThreadLocalUtils.java:84) at org.openecomp.sdc.be.filters.ThreadLocalUtils.setUserContextFromDB(ThreadLocalUtils.java:77) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:70) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:36.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:13:36.427Z [qtp2045766957-14] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getNode(JanusGraphGenericDao.java:317) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.validateUserExists(UserAdminOperation.java:129) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.getUserData(UserAdminOperation.java:77) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.getUserData(UserAdminOperation.java:69) at org.openecomp.sdc.be.user.UserBusinessLogic.getUser(UserBusinessLogic.java:71) at org.openecomp.sdc.be.filters.ThreadLocalUtils.updateUserContext(ThreadLocalUtils.java:84) at org.openecomp.sdc.be.filters.ThreadLocalUtils.setUserContextFromDB(ThreadLocalUtils.java:77) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:70) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:36.830Z [qtp2045766957-14] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getNode(JanusGraphGenericDao.java:317) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.validateUserExists(UserAdminOperation.java:129) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.getUserData(UserAdminOperation.java:77) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.getUserData(UserAdminOperation.java:69) at org.openecomp.sdc.be.user.UserBusinessLogic.getUser(UserBusinessLogic.java:71) at org.openecomp.sdc.be.filters.ThreadLocalUtils.updateUserContext(ThreadLocalUtils.java:84) at org.openecomp.sdc.be.filters.ThreadLocalUtils.setUserContextFromDB(ThreadLocalUtils.java:77) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:70) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:37.178Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:37.231Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:37.232Z [qtp2045766957-14] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getNode(JanusGraphGenericDao.java:317) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.validateUserExists(UserAdminOperation.java:129) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.getUserData(UserAdminOperation.java:77) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.getUserData(UserAdminOperation.java:69) at org.openecomp.sdc.be.user.UserBusinessLogic.getUser(UserBusinessLogic.java:71) at org.openecomp.sdc.be.filters.ThreadLocalUtils.updateUserContext(ThreadLocalUtils.java:84) at org.openecomp.sdc.be.filters.ThreadLocalUtils.setUserContextFromDB(ThreadLocalUtils.java:77) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:70) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:37.283Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:37.386Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:37.647Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:37.647Z [qtp2045766957-14] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getNode(JanusGraphGenericDao.java:317) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.validateUserExists(UserAdminOperation.java:129) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.getUserData(UserAdminOperation.java:77) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.getUserData(UserAdminOperation.java:69) at org.openecomp.sdc.be.user.UserBusinessLogic.getUser(UserBusinessLogic.java:71) at org.openecomp.sdc.be.filters.ThreadLocalUtils.updateUserContext(ThreadLocalUtils.java:84) at org.openecomp.sdc.be.filters.ThreadLocalUtils.setUserContextFromDB(ThreadLocalUtils.java:77) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:70) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:37.751Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a711fd58-3f7c-4444-910b-b6bc9774eef6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:13:37.751Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a711fd58-3f7c-4444-910b-b6bc9774eef6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5dc062a8 2021-08-11T16:13:37.751Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a711fd58-3f7c-4444-910b-b6bc9774eef6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:13:37.751Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a711fd58-3f7c-4444-910b-b6bc9774eef6 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:13:37.850Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:38.050Z [qtp2045766957-14] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getNode(JanusGraphGenericDao.java:317) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.validateUserExists(UserAdminOperation.java:129) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.getUserData(UserAdminOperation.java:77) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.getUserData(UserAdminOperation.java:69) at org.openecomp.sdc.be.user.UserBusinessLogic.getUser(UserBusinessLogic.java:71) at org.openecomp.sdc.be.filters.ThreadLocalUtils.updateUserContext(ThreadLocalUtils.java:84) at org.openecomp.sdc.be.filters.ThreadLocalUtils.setUserContextFromDB(ThreadLocalUtils.java:77) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:70) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:38.052Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:38.455Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:38.462Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:38.548Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:38.647Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:38.750Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:38.853Z [qtp2045766957-14] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getNode(JanusGraphGenericDao.java:317) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.validateUserExists(UserAdminOperation.java:129) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.getUserData(UserAdminOperation.java:77) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.getUserData(UserAdminOperation.java:69) at org.openecomp.sdc.be.user.UserBusinessLogic.getUser(UserBusinessLogic.java:71) at org.openecomp.sdc.be.filters.ThreadLocalUtils.updateUserContext(ThreadLocalUtils.java:84) at org.openecomp.sdc.be.filters.ThreadLocalUtils.setUserContextFromDB(ThreadLocalUtils.java:77) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:70) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:38.857Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:38.953Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:39.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:13:39.260Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:39.356Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:39.657Z [qtp2045766957-14] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getNode(JanusGraphGenericDao.java:317) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.validateUserExists(UserAdminOperation.java:129) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.getUserData(UserAdminOperation.java:77) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.getUserData(UserAdminOperation.java:69) at org.openecomp.sdc.be.user.UserBusinessLogic.getUser(UserBusinessLogic.java:71) at org.openecomp.sdc.be.filters.ThreadLocalUtils.updateUserContext(ThreadLocalUtils.java:84) at org.openecomp.sdc.be.filters.ThreadLocalUtils.setUserContextFromDB(ThreadLocalUtils.java:77) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:70) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:39.662Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:40.066Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:40.159Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:40.460Z [qtp2045766957-14] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getNode(JanusGraphGenericDao.java:317) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.validateUserExists(UserAdminOperation.java:129) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.getUserData(UserAdminOperation.java:77) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.getUserData(UserAdminOperation.java:69) at org.openecomp.sdc.be.user.UserBusinessLogic.getUser(UserBusinessLogic.java:71) at org.openecomp.sdc.be.filters.ThreadLocalUtils.updateUserContext(ThreadLocalUtils.java:84) at org.openecomp.sdc.be.filters.ThreadLocalUtils.setUserContextFromDB(ThreadLocalUtils.java:77) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:70) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:40.468Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:40.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52c2c6cd-0170-4b26-91c5-3f95cee71be8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:13:40.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52c2c6cd-0170-4b26-91c5-3f95cee71be8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3964cef1 2021-08-11T16:13:40.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52c2c6cd-0170-4b26-91c5-3f95cee71be8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:13:40.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52c2c6cd-0170-4b26-91c5-3f95cee71be8 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:13:40.871Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T16:12:57.862205Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:40.964Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:41.767Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:42.063Z [qtp2045766957-14] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getNode(JanusGraphGenericDao.java:317) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.validateUserExists(UserAdminOperation.java:129) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.getUserData(UserAdminOperation.java:77) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.getUserData(UserAdminOperation.java:69) at org.openecomp.sdc.be.user.UserBusinessLogic.getUser(UserBusinessLogic.java:71) at org.openecomp.sdc.be.filters.ThreadLocalUtils.updateUserContext(ThreadLocalUtils.java:84) at org.openecomp.sdc.be.filters.ThreadLocalUtils.setUserContextFromDB(ThreadLocalUtils.java:77) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:70) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:42.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:13:43.371Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:43.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=89258a53-7f6d-46b8-9fab-c96c197991d2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:13:43.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=89258a53-7f6d-46b8-9fab-c96c197991d2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@39250070 2021-08-11T16:13:43.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=89258a53-7f6d-46b8-9fab-c96c197991d2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:13:43.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=89258a53-7f6d-46b8-9fab-c96c197991d2 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:13:43.666Z [qtp2045766957-14] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getVertexByProperty(JanusGraphGenericDao.java:617) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphGenericDao.getNode(JanusGraphGenericDao.java:317) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.validateUserExists(UserAdminOperation.java:129) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.getUserData(UserAdminOperation.java:77) at org.openecomp.sdc.be.model.operations.impl.UserAdminOperation.getUserData(UserAdminOperation.java:69) at org.openecomp.sdc.be.user.UserBusinessLogic.getUser(UserBusinessLogic.java:71) at org.openecomp.sdc.be.filters.ThreadLocalUtils.updateUserContext(ThreadLocalUtils.java:84) at org.openecomp.sdc.be.filters.ThreadLocalUtils.setUserContextFromDB(ThreadLocalUtils.java:77) at org.openecomp.sdc.be.filters.GatewayFilter.doFilter(GatewayFilter.java:70) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1630) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:567) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1610) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1377) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:507) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1580) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1292) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:191) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.Server.handle(Server.java:501) at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383) at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395) at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ... 1 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:44.975Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:13:45.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:13:45.364Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=dc9a1cf8-f0f3-43e6-89c4-d003a344df60 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:13:45.364233Z InvocationID=96776920-18d4-4c61-97c2-40aa50bb7080 LogTimestamp=2021-08-11T16:13:45.364356Z Entering 2021-08-11T16:13:45.365Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=dc9a1cf8-f0f3-43e6-89c4-d003a344df60 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:13:45.364233Z InvocationID=96776920-18d4-4c61-97c2-40aa50bb7080 LogTimestamp=2021-08-11T16:13:45.364356Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:13:45.366Z [qtp2045766957-14] ERROR o.o.s.b.f.ComponentsAvailabilityFilter RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2021-08-11T16:13:45.365Z InvocationID=44821252-c8f8-4dee-b65a-cf8dd11f3098 ErrorCode=500 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null Components Availability Filter Failed - ES/Cassandra is DOWN 2021-08-11T16:13:45.522Z [qtp2045766957-14] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=4 EndTimestamp=2021-08-11T16:13:45.369Z Severity=0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=ERROR TargetEntity=null RemoteHost=10.242.110.138 ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-08-11T16:13:45.365Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=44821252-c8f8-4dee-b65a-cf8dd11f3098 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@5eac27fe 2021-08-11T16:13:45.522Z [qtp2045766957-14] ERROR o.o.s.b.f.ComponentsAvailabilityFilter ResponseCode=500 RequestID=null InstanceID=null ElapsedTime=4 EndTimestamp=2021-08-11T16:13:45.369Z Severity=0 PartnerName=jh0003 auditOn=false ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=ERROR TargetEntity=null RemoteHost=10.242.110.138 ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=null TargetServiceName=null EntryTimestamp=2021-08-11T16:13:45.365Z ClassName=org.openecomp.sdc.be.filters.ComponentsAvailabilityFilter InvocationID=44821252-c8f8-4dee-b65a-cf8dd11f3098 ErrorCode=500 ResponseDesc=Internal Server Error org.openecomp.sdc.exception.ResponseFormat$RequestErrorWrapper@5eac27fe 2021-08-11T16:13:45.524Z [qtp2045766957-14] WARN o.onap.logging.filter.base.MDCSetup ResponseCode=500 ServerIPAddress=10.242.24.223 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:13:45.524405Z ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=ERROR ResponseDesc=Internal Server Error Unable to calculate elapsed time due to error: text 2021-08-11T16:13:45.524Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=500 ServerIPAddress=10.242.24.223 ErrorDesc=Internal Server Error ErrorCode=500 LogTimestamp=2021-08-11T16:13:45.524405Z ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=ERROR ResponseDesc=Internal Server Error Exiting. 2021-08-11T16:13:45.524Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="500" 2021-08-11T16:13:46.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b1db75b-0f00-4600-b3e5-c70697c1086c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:13:46.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b1db75b-0f00-4600-b3e5-c70697c1086c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1afe00cc 2021-08-11T16:13:46.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b1db75b-0f00-4600-b3e5-c70697c1086c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:13:46.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b1db75b-0f00-4600-b3e5-c70697c1086c PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:13:47.157Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=2b1db75b-0f00-4600-b3e5-c70697c1086c ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:13:47.157Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=2b1db75b-0f00-4600-b3e5-c70697c1086c ErrorCategory=ERROR ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-08-11T16:13:48.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:13:48.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:13:48.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:13:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:13:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:13:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:13:49.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=59ae4151-eed9-453d-9462-53a6c385b140 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:13:49.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=59ae4151-eed9-453d-9462-53a6c385b140 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@44f5e419 2021-08-11T16:13:49.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=59ae4151-eed9-453d-9462-53a6c385b140 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:13:49.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=59ae4151-eed9-453d-9462-53a6c385b140 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:13:50.271Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-08-11T16:13:52.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7a8f969-3e55-4ccb-8cd0-7a63908c0ca0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:13:52.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7a8f969-3e55-4ccb-8cd0-7a63908c0ca0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@205c955a 2021-08-11T16:13:52.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7a8f969-3e55-4ccb-8cd0-7a63908c0ca0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:13:52.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7a8f969-3e55-4ccb-8cd0-7a63908c0ca0 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:13:53.055Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=f7a8f969-3e55-4ccb-8cd0-7a63908c0ca0 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:13:53.055Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=f7a8f969-3e55-4ccb-8cd0-7a63908c0ca0 ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-08-11T16:13:55.570Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=ac322d88-f11e-4013-880b-1ef1058d2e9c InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:13:55.569908Z InvocationID=a8d9e3c1-89bc-4690-af97-dbee25ea8c87 LogTimestamp=2021-08-11T16:13:55.570026Z Entering 2021-08-11T16:13:55.570Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=ac322d88-f11e-4013-880b-1ef1058d2e9c InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:13:55.569908Z InvocationID=a8d9e3c1-89bc-4690-af97-dbee25ea8c87 LogTimestamp=2021-08-11T16:13:55.570026Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:13:55.595Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8695102d-2454-4198-9dcd-d408ec7c01b6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:13:55.595Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8695102d-2454-4198-9dcd-d408ec7c01b6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4b626864 2021-08-11T16:13:55.595Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8695102d-2454-4198-9dcd-d408ec7c01b6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:13:55.595Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8695102d-2454-4198-9dcd-d408ec7c01b6 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:13:56.985Z [qtp2045766957-17] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=Import Resource VRFEntry.yml ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2021-08-11T16:13:55.571Z InvocationID=2e773410-e84e-4b62-95dd-642331a0b9b9 ErrorCode=500 Unexpected error during operation 2021-08-11T16:13:56.994Z [qtp2045766957-17] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "" INVARIANT_UUID = "" PREV_VERSION = "1.0" CURR_VERSION = "" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "" STATUS = "409" DESC = "SVC4083: Error: Version of this VRF Entry Configuration was already promoted." TOSCA_NODE_TYPE = "" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=Upload Resource ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2021-08-11T16:13:55.571Z InvocationID=2e773410-e84e-4b62-95dd-642331a0b9b9 ErrorCode=900 Unexpected error during BE REST API execution 2021-08-11T16:13:56.995Z [qtp2045766957-17] ERROR o.o.s.b.s.ResourceUploadServlet AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "" INVARIANT_UUID = "" PREV_VERSION = "1.0" CURR_VERSION = "" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "" STATUS = "409" DESC = "SVC4083: Error: Version of this VRF Entry Configuration was already promoted." TOSCA_NODE_TYPE = "" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=Upload Resource ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2021-08-11T16:13:55.571Z InvocationID=2e773410-e84e-4b62-95dd-642331a0b9b9 ErrorCode=900 #uploadMultipart('multipart', /tmp/rep11752156980170405013tmp, 'form-data; filename="VRFEntry.zip"; name="resourceZip"', '{"payloadName": "VRFEntry.yml", "contactId": "jh0003", "name": "VRF Entry Configuration", "description": "VRF Entry configuration object", "resourceIconPath": "pmc", "resourceType": "Configuration", "categories": [{"name": "Configuration", "subcategories": [{"name": "Configuration"}]}], "tags": ["VRF Entry Configuration"]}', 'Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/catalog/upload/multipart?createNewVersion=true)@52a4e9a9', 'jh0003', true): thrown org.openecomp.sdc.be.components.impl.exceptions.ByActionStatusComponentException out of org.openecomp.sdc.be.components.impl.ResourceImportManager#importCertifiedResource[184] in 1s 2021-08-11T16:13:56.996Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VRF Entry Configuration" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "" INVARIANT_UUID = "" PREV_VERSION = "1.0" CURR_VERSION = "" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "" STATUS = "409" DESC = "SVC4083: Error: Version of this VRF Entry Configuration was already promoted." TOSCA_NODE_TYPE = "" ResponseCode=409 RequestID=null ElapsedTime=1424 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=ERROR TargetEntity=Upload Resource ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null ErrorDesc=Conflict EntryTimestamp=2021-08-11T16:13:55.571Z InvocationID=2e773410-e84e-4b62-95dd-642331a0b9b9 ErrorCode=409 LogTimestamp=2021-08-11T16:13:56.995915Z ResponseDesc=Conflict Exiting. 2021-08-11T16:13:56.996Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="409" 2021-08-11T16:13:57.069Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=5c30a3f2-c538-4938-a814-1e42afa0cedc InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:13:57.068857Z InvocationID=ba1124ac-7ae3-462b-a576-527f23a0afee LogTimestamp=2021-08-11T16:13:57.068917Z Entering 2021-08-11T16:13:57.069Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=5c30a3f2-c538-4938-a814-1e42afa0cedc InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:13:57.068857Z InvocationID=ba1124ac-7ae3-462b-a576-527f23a0afee LogTimestamp=2021-08-11T16:13:57.068917Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:13:58.382Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:13:57.069Z InvocationID=519742f7-7699-4248-a98a-1e0cd49ecc08 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:13:58.382Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:13:57.069Z InvocationID=519742f7-7699-4248-a98a-1e0cd49ecc08 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:13:58.382Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:13:57.069Z InvocationID=519742f7-7699-4248-a98a-1e0cd49ecc08 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:13:58.382Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:13:57.069Z InvocationID=519742f7-7699-4248-a98a-1e0cd49ecc08 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:13:58.484Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:13:57.069Z InvocationID=519742f7-7699-4248-a98a-1e0cd49ecc08 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Subinterfacev2 2021-08-11T16:14:00.150Z [qtp2045766957-14] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "subInterfaceV2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "87aa2b14-fd1d-42d8-a855-a4549482db92" INVARIANT_UUID = "8d97c91f-918c-454d-8c89-28a0e24d0118" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.network.v2.SubInterface" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=87aa2b14-fd1d-42d8-a855-a4549482db92 EntryTimestamp=2021-08-11T16:13:57.069Z InvocationID=519742f7-7699-4248-a98a-1e0cd49ecc08 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:14:00.151Z [qtp2045766957-14] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "subInterfaceV2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "87aa2b14-fd1d-42d8-a855-a4549482db92" INVARIANT_UUID = "8d97c91f-918c-454d-8c89-28a0e24d0118" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.network.v2.SubInterface" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=87aa2b14-fd1d-42d8-a855-a4549482db92 TargetServiceName=null EntryTimestamp=2021-08-11T16:13:57.069Z InvocationID=519742f7-7699-4248-a98a-1e0cd49ecc08 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:14:01.771Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "subInterfaceV2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "87aa2b14-fd1d-42d8-a855-a4549482db92" INVARIANT_UUID = "8d97c91f-918c-454d-8c89-28a0e24d0118" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.network.v2.SubInterface" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=87aa2b14-fd1d-42d8-a855-a4549482db92 TargetServiceName=null EntryTimestamp=2021-08-11T16:13:57.069Z InvocationID=519742f7-7699-4248-a98a-1e0cd49ecc08 ErrorCode=500 lock resource with id 449b0926-ae64-43a5-8af9-d374659fff13 2021-08-11T16:14:03.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:14:03.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:14:04.258Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:14:04.264Z [qtp2045766957-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "subInterfaceV2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "87aa2b14-fd1d-42d8-a855-a4549482db92" INVARIANT_UUID = "8d97c91f-918c-454d-8c89-28a0e24d0118" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "449b0926-ae64-43a5-8af9-d374659fff13.assettoscatemplate" CURR_ARTIFACT_UUID = "449b0926-ae64-43a5-8af9-d374659fff13.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Subinterfacev2-template.yml,0,449b0926-ae64-43a5-8af9-d374659fff13.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=92c44785-240b-42c5-91f3-70bde5a055c8 TargetServiceName=null EntryTimestamp=2021-08-11T16:13:57.069Z InvocationID=519742f7-7699-4248-a98a-1e0cd49ecc08 ErrorCode=500 Starting to create Csar for component subInterfaceV2 2021-08-11T16:14:04.460Z [qtp2045766957-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "subInterfaceV2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "87aa2b14-fd1d-42d8-a855-a4549482db92" INVARIANT_UUID = "8d97c91f-918c-454d-8c89-28a0e24d0118" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "449b0926-ae64-43a5-8af9-d374659fff13.assettoscatemplate" CURR_ARTIFACT_UUID = "449b0926-ae64-43a5-8af9-d374659fff13.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Subinterfacev2-template.yml,0,449b0926-ae64-43a5-8af9-d374659fff13.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=92c44785-240b-42c5-91f3-70bde5a055c8 TargetServiceName=null EntryTimestamp=2021-08-11T16:13:57.069Z InvocationID=519742f7-7699-4248-a98a-1e0cd49ecc08 ErrorCode=500 Ended create Csar for component subInterfaceV2 2021-08-11T16:14:04.461Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "subInterfaceV2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "87aa2b14-fd1d-42d8-a855-a4549482db92" INVARIANT_UUID = "8d97c91f-918c-454d-8c89-28a0e24d0118" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "449b0926-ae64-43a5-8af9-d374659fff13.assettoscatemplate" CURR_ARTIFACT_UUID = "449b0926-ae64-43a5-8af9-d374659fff13.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Subinterfacev2-template.yml,0,449b0926-ae64-43a5-8af9-d374659fff13.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=92c44785-240b-42c5-91f3-70bde5a055c8 TargetServiceName=null EntryTimestamp=2021-08-11T16:13:57.069Z InvocationID=519742f7-7699-4248-a98a-1e0cd49ecc08 ErrorCode=500 lock resource with id 449b0926-ae64-43a5-8af9-d374659fff13 2021-08-11T16:14:07.281Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "subInterfaceV2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "87aa2b14-fd1d-42d8-a855-a4549482db92" INVARIANT_UUID = "8d97c91f-918c-454d-8c89-28a0e24d0118" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "449b0926-ae64-43a5-8af9-d374659fff13.assettoscacsar" CURR_ARTIFACT_UUID = "449b0926-ae64-43a5-8af9-d374659fff13.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Subinterfacev2-csar.csar,0,449b0926-ae64-43a5-8af9-d374659fff13.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=10211 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b9d8724c-f503-420d-9483-114f184ab5d2 TargetServiceName=null EntryTimestamp=2021-08-11T16:13:57.069Z InvocationID=519742f7-7699-4248-a98a-1e0cd49ecc08 ErrorCode=500 LogTimestamp=2021-08-11T16:14:07.280880Z ResponseDesc=Created Exiting. 2021-08-11T16:14:07.281Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:14:07.366Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=ce060e60-3d4b-412a-b0f9-eeed00f82480 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:14:07.366217Z InvocationID=4de8a2a5-6885-4ddc-a665-6679016f831e LogTimestamp=2021-08-11T16:14:07.366286Z Entering 2021-08-11T16:14:07.366Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=ce060e60-3d4b-412a-b0f9-eeed00f82480 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:14:07.366217Z InvocationID=4de8a2a5-6885-4ddc-a665-6679016f831e LogTimestamp=2021-08-11T16:14:07.366286Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:14:07.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d3b68789-f221-4321-baa4-1771d628f8c9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:14:07.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d3b68789-f221-4321-baa4-1771d628f8c9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5dc4f801 2021-08-11T16:14:07.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d3b68789-f221-4321-baa4-1771d628f8c9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:14:07.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d3b68789-f221-4321-baa4-1771d628f8c9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:14:09.262Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=d3b68789-f221-4321-baa4-1771d628f8c9 ErrorCategory=INFO ServiceName=BE Component CASSANDRA state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component CASSANDRA state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:14:09.262Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=d3b68789-f221-4321-baa4-1771d628f8c9 ErrorCategory=INFO ServiceName=BE Component CASSANDRA state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:14:10.461Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e817c16a-699a-4bc8-a91e-e2052a538e6f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:14:10.461Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e817c16a-699a-4bc8-a91e-e2052a538e6f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2327db97 2021-08-11T16:14:10.461Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e817c16a-699a-4bc8-a91e-e2052a538e6f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:14:10.461Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e817c16a-699a-4bc8-a91e-e2052a538e6f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:14:10.951Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:14:07.367Z InvocationID=32741be7-abf5-4d8b-bf0f-25b68af19f05 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:14:10.952Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:14:07.367Z InvocationID=32741be7-abf5-4d8b-bf0f-25b68af19f05 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:14:10.952Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:14:07.367Z InvocationID=32741be7-abf5-4d8b-bf0f-25b68af19f05 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:14:10.952Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:14:07.367Z InvocationID=32741be7-abf5-4d8b-bf0f-25b68af19f05 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:14:11.077Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:14:07.367Z InvocationID=32741be7-abf5-4d8b-bf0f-25b68af19f05 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Contrailv2vlansubinterfacev2 2021-08-11T16:14:12.351Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f0c0810-630d-4d1e-a09b-38ba165d9bf5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:14:12.351Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f0c0810-630d-4d1e-a09b-38ba165d9bf5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7bdeb6df 2021-08-11T16:14:12.351Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f0c0810-630d-4d1e-a09b-38ba165d9bf5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:14:12.351Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f0c0810-630d-4d1e-a09b-38ba165d9bf5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:14:14.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e27449bf-4d54-4c32-9f18-22eca8cee1aa PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:14:14.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e27449bf-4d54-4c32-9f18-22eca8cee1aa PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1205d55 2021-08-11T16:14:14.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e27449bf-4d54-4c32-9f18-22eca8cee1aa PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:14:14.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e27449bf-4d54-4c32-9f18-22eca8cee1aa PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:14:15.263Z [qtp2045766957-18] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "contrailV2VLANSubInterfaceV2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "abb84b33-7c6d-4296-bdc1-4e2e64278580" INVARIANT_UUID = "4b7762f2-1d88-4cb6-8ce8-e43f830b7d49" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.heat.network.v2.contrailV2.VLANSubInterface" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=abb84b33-7c6d-4296-bdc1-4e2e64278580 EntryTimestamp=2021-08-11T16:14:07.367Z InvocationID=32741be7-abf5-4d8b-bf0f-25b68af19f05 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:14:15.264Z [qtp2045766957-18] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "contrailV2VLANSubInterfaceV2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "abb84b33-7c6d-4296-bdc1-4e2e64278580" INVARIANT_UUID = "4b7762f2-1d88-4cb6-8ce8-e43f830b7d49" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.heat.network.v2.contrailV2.VLANSubInterface" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=abb84b33-7c6d-4296-bdc1-4e2e64278580 TargetServiceName=null EntryTimestamp=2021-08-11T16:14:07.367Z InvocationID=32741be7-abf5-4d8b-bf0f-25b68af19f05 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:14:17.066Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=335f1c5d-2775-4968-8956-c5b1a514e1ab PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:14:17.067Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=335f1c5d-2775-4968-8956-c5b1a514e1ab PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4ddacf3f 2021-08-11T16:14:17.067Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=335f1c5d-2775-4968-8956-c5b1a514e1ab PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:14:17.067Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=335f1c5d-2775-4968-8956-c5b1a514e1ab PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:14:17.366Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "contrailV2VLANSubInterfaceV2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "abb84b33-7c6d-4296-bdc1-4e2e64278580" INVARIANT_UUID = "4b7762f2-1d88-4cb6-8ce8-e43f830b7d49" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.nodes.heat.network.v2.contrailV2.VLANSubInterface" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=abb84b33-7c6d-4296-bdc1-4e2e64278580 TargetServiceName=null EntryTimestamp=2021-08-11T16:14:07.367Z InvocationID=32741be7-abf5-4d8b-bf0f-25b68af19f05 ErrorCode=500 lock resource with id 246ef4af-6d27-491c-bc4d-7ddc970e63d6 2021-08-11T16:14:18.291Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8fa4c12-eb6b-4283-9f99-603109c95649 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:14:18.291Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8fa4c12-eb6b-4283-9f99-603109c95649 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@503c5e70 2021-08-11T16:14:18.291Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8fa4c12-eb6b-4283-9f99-603109c95649 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:14:18.291Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8fa4c12-eb6b-4283-9f99-603109c95649 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:14:18.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:14:18.317Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:14:19.871Z [qtp2045766957-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "contrailV2VLANSubInterfaceV2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "abb84b33-7c6d-4296-bdc1-4e2e64278580" INVARIANT_UUID = "4b7762f2-1d88-4cb6-8ce8-e43f830b7d49" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "246ef4af-6d27-491c-bc4d-7ddc970e63d6.assettoscatemplate" CURR_ARTIFACT_UUID = "246ef4af-6d27-491c-bc4d-7ddc970e63d6.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailv2vlansubinterfacev2-template.yml,0,246ef4af-6d27-491c-bc4d-7ddc970e63d6.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=4fd29710-b30a-4a64-b4f1-792b3e66b63a TargetServiceName=null EntryTimestamp=2021-08-11T16:14:07.367Z InvocationID=32741be7-abf5-4d8b-bf0f-25b68af19f05 ErrorCode=500 Starting to create Csar for component contrailV2VLANSubInterfaceV2 2021-08-11T16:14:19.967Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d3814073-a116-404d-b761-5bc2ea781db8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:14:19.967Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d3814073-a116-404d-b761-5bc2ea781db8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5e5a72e6 2021-08-11T16:14:19.967Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d3814073-a116-404d-b761-5bc2ea781db8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:14:19.967Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d3814073-a116-404d-b761-5bc2ea781db8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:14:20.085Z [qtp2045766957-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "contrailV2VLANSubInterfaceV2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "abb84b33-7c6d-4296-bdc1-4e2e64278580" INVARIANT_UUID = "4b7762f2-1d88-4cb6-8ce8-e43f830b7d49" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "246ef4af-6d27-491c-bc4d-7ddc970e63d6.assettoscatemplate" CURR_ARTIFACT_UUID = "246ef4af-6d27-491c-bc4d-7ddc970e63d6.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailv2vlansubinterfacev2-template.yml,0,246ef4af-6d27-491c-bc4d-7ddc970e63d6.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=4fd29710-b30a-4a64-b4f1-792b3e66b63a TargetServiceName=null EntryTimestamp=2021-08-11T16:14:07.367Z InvocationID=32741be7-abf5-4d8b-bf0f-25b68af19f05 ErrorCode=500 Ended create Csar for component contrailV2VLANSubInterfaceV2 2021-08-11T16:14:20.085Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "contrailV2VLANSubInterfaceV2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "abb84b33-7c6d-4296-bdc1-4e2e64278580" INVARIANT_UUID = "4b7762f2-1d88-4cb6-8ce8-e43f830b7d49" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "246ef4af-6d27-491c-bc4d-7ddc970e63d6.assettoscatemplate" CURR_ARTIFACT_UUID = "246ef4af-6d27-491c-bc4d-7ddc970e63d6.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Contrailv2vlansubinterfacev2-template.yml,0,246ef4af-6d27-491c-bc4d-7ddc970e63d6.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=4fd29710-b30a-4a64-b4f1-792b3e66b63a TargetServiceName=null EntryTimestamp=2021-08-11T16:14:07.367Z InvocationID=32741be7-abf5-4d8b-bf0f-25b68af19f05 ErrorCode=500 lock resource with id 246ef4af-6d27-491c-bc4d-7ddc970e63d6 2021-08-11T16:14:20.887Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a858eaa2-6e8d-4654-b159-2934ca652f2f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:14:20.887Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a858eaa2-6e8d-4654-b159-2934ca652f2f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5d93468d 2021-08-11T16:14:20.888Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a858eaa2-6e8d-4654-b159-2934ca652f2f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:14:20.888Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a858eaa2-6e8d-4654-b159-2934ca652f2f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:14:22.020Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "contrailV2VLANSubInterfaceV2" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "abb84b33-7c6d-4296-bdc1-4e2e64278580" INVARIANT_UUID = "4b7762f2-1d88-4cb6-8ce8-e43f830b7d49" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "246ef4af-6d27-491c-bc4d-7ddc970e63d6.assettoscacsar" CURR_ARTIFACT_UUID = "246ef4af-6d27-491c-bc4d-7ddc970e63d6.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Contrailv2vlansubinterfacev2-csar.csar,0,246ef4af-6d27-491c-bc4d-7ddc970e63d6.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=14653 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=05dafad9-6313-4516-9aea-33f371007077 TargetServiceName=null EntryTimestamp=2021-08-11T16:14:07.367Z InvocationID=32741be7-abf5-4d8b-bf0f-25b68af19f05 ErrorCode=500 LogTimestamp=2021-08-11T16:14:22.020241Z ResponseDesc=Created Exiting. 2021-08-11T16:14:22.021Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:14:22.070Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=48cd8ad9-bbf5-4938-915c-2ecb3722f83c InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:14:22.069830Z InvocationID=5cec4d22-368b-406a-9499-545c75613523 LogTimestamp=2021-08-11T16:14:22.069904Z Entering 2021-08-11T16:14:22.070Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=48cd8ad9-bbf5-4938-915c-2ecb3722f83c InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:14:22.069830Z InvocationID=5cec4d22-368b-406a-9499-545c75613523 LogTimestamp=2021-08-11T16:14:22.069904Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:14:23.090Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=259be332-be77-4cdb-abca-8b0c9c620f02 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:14:23.090Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=259be332-be77-4cdb-abca-8b0c9c620f02 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@46abd937 2021-08-11T16:14:23.090Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=259be332-be77-4cdb-abca-8b0c9c620f02 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:14:23.090Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=259be332-be77-4cdb-abca-8b0c9c620f02 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:14:23.269Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:14:22.070Z InvocationID=ebf80ead-7fe6-4c65-82fe-353136b62a03 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:14:23.270Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:14:22.070Z InvocationID=ebf80ead-7fe6-4c65-82fe-353136b62a03 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:14:23.270Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:14:22.070Z InvocationID=ebf80ead-7fe6-4c65-82fe-353136b62a03 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:14:23.270Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:14:22.070Z InvocationID=ebf80ead-7fe6-4c65-82fe-353136b62a03 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:14:23.287Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:14:22.070Z InvocationID=ebf80ead-7fe6-4c65-82fe-353136b62a03 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name FabricConfiguration 2021-08-11T16:14:25.319Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Fabric Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "d821e7ed-2a15-4372-973e-37cc205bfa12" INVARIANT_UUID = "f09c2c86-3a3d-4928-88a3-3f50cb2b994f" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.nodes.FabricConfiguration" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d821e7ed-2a15-4372-973e-37cc205bfa12 EntryTimestamp=2021-08-11T16:14:22.070Z InvocationID=ebf80ead-7fe6-4c65-82fe-353136b62a03 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id 970af896-39f8-4e9c-92b7-fd9a17664327 2021-08-11T16:14:25.818Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b2e5dcf-b707-4f75-b0f1-ae3931e4a1a7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:14:25.818Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b2e5dcf-b707-4f75-b0f1-ae3931e4a1a7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@74d6d04c 2021-08-11T16:14:25.818Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b2e5dcf-b707-4f75-b0f1-ae3931e4a1a7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:14:25.818Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b2e5dcf-b707-4f75-b0f1-ae3931e4a1a7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:14:27.277Z [qtp2045766957-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Fabric Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "d821e7ed-2a15-4372-973e-37cc205bfa12" INVARIANT_UUID = "f09c2c86-3a3d-4928-88a3-3f50cb2b994f" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "970af896-39f8-4e9c-92b7-fd9a17664327.assettoscatemplate" CURR_ARTIFACT_UUID = "970af896-39f8-4e9c-92b7-fd9a17664327.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-FabricConfiguration-template.yml,0,970af896-39f8-4e9c-92b7-fd9a17664327.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=12bf934d-a519-45b9-876d-a2b14f5aef42 EntryTimestamp=2021-08-11T16:14:22.070Z InvocationID=ebf80ead-7fe6-4c65-82fe-353136b62a03 PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Starting to create Csar for component Fabric Configuration 2021-08-11T16:14:27.471Z [qtp2045766957-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Fabric Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "d821e7ed-2a15-4372-973e-37cc205bfa12" INVARIANT_UUID = "f09c2c86-3a3d-4928-88a3-3f50cb2b994f" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "970af896-39f8-4e9c-92b7-fd9a17664327.assettoscatemplate" CURR_ARTIFACT_UUID = "970af896-39f8-4e9c-92b7-fd9a17664327.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-FabricConfiguration-template.yml,0,970af896-39f8-4e9c-92b7-fd9a17664327.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=12bf934d-a519-45b9-876d-a2b14f5aef42 EntryTimestamp=2021-08-11T16:14:22.070Z InvocationID=ebf80ead-7fe6-4c65-82fe-353136b62a03 Ended create Csar for component Fabric Configuration 2021-08-11T16:14:27.471Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Fabric Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "d821e7ed-2a15-4372-973e-37cc205bfa12" INVARIANT_UUID = "f09c2c86-3a3d-4928-88a3-3f50cb2b994f" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "970af896-39f8-4e9c-92b7-fd9a17664327.assettoscatemplate" CURR_ARTIFACT_UUID = "970af896-39f8-4e9c-92b7-fd9a17664327.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-FabricConfiguration-template.yml,0,970af896-39f8-4e9c-92b7-fd9a17664327.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=12bf934d-a519-45b9-876d-a2b14f5aef42 EntryTimestamp=2021-08-11T16:14:22.070Z InvocationID=ebf80ead-7fe6-4c65-82fe-353136b62a03 lock resource with id 970af896-39f8-4e9c-92b7-fd9a17664327 2021-08-11T16:14:28.752Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3d5efbb8-5d40-4004-a339-15170fab1d3b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:14:28.752Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3d5efbb8-5d40-4004-a339-15170fab1d3b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@79965172 2021-08-11T16:14:28.752Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3d5efbb8-5d40-4004-a339-15170fab1d3b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:14:28.752Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3d5efbb8-5d40-4004-a339-15170fab1d3b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:14:29.682Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Fabric Configuration" RESOURCE_TYPE = "Configuration" SERVICE_INSTANCE_ID = "d821e7ed-2a15-4372-973e-37cc205bfa12" INVARIANT_UUID = "f09c2c86-3a3d-4928-88a3-3f50cb2b994f" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "970af896-39f8-4e9c-92b7-fd9a17664327.assettoscacsar" CURR_ARTIFACT_UUID = "970af896-39f8-4e9c-92b7-fd9a17664327.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-FabricConfiguration-csar.csar,0,970af896-39f8-4e9c-92b7-fd9a17664327.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=7612 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c8d45587-52b9-426a-bdc9-f33916911542 EntryTimestamp=2021-08-11T16:14:22.070Z InvocationID=ebf80ead-7fe6-4c65-82fe-353136b62a03 LogTimestamp=2021-08-11T16:14:29.682565Z ResponseDesc=Created Exiting. 2021-08-11T16:14:29.683Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:14:29.756Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=6c45cd22-d048-4f90-b0b8-579aa14f2aec InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:14:29.756334Z InvocationID=e2fa0a0b-11da-4098-9555-86905c327da5 LogTimestamp=2021-08-11T16:14:29.756408Z Entering 2021-08-11T16:14:29.756Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=6c45cd22-d048-4f90-b0b8-579aa14f2aec InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:14:29.756334Z InvocationID=e2fa0a0b-11da-4098-9555-86905c327da5 LogTimestamp=2021-08-11T16:14:29.756408Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:14:30.459Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:14:29.757Z InvocationID=cce9ce37-ee75-482e-9d56-1ceea3075dbc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:14:30.459Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:14:29.757Z InvocationID=cce9ce37-ee75-482e-9d56-1ceea3075dbc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:14:30.459Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:14:29.757Z InvocationID=cce9ce37-ee75-482e-9d56-1ceea3075dbc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:14:30.459Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:14:29.757Z InvocationID=cce9ce37-ee75-482e-9d56-1ceea3075dbc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:14:30.554Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:14:29.757Z InvocationID=cce9ce37-ee75-482e-9d56-1ceea3075dbc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name VlUnderlayvpn 2021-08-11T16:14:31.898Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=493abba4-54ac-49f4-a03c-a1af5b1536f0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:14:31.899Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=493abba4-54ac-49f4-a03c-a1af5b1536f0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3ef6c7e3 2021-08-11T16:14:31.899Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=493abba4-54ac-49f4-a03c-a1af5b1536f0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:14:31.899Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=493abba4-54ac-49f4-a03c-a1af5b1536f0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:14:32.474Z [qtp2045766957-18] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VL UNDERLAYVPN" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "6172e2ce-72ca-43eb-8727-e5694f364023" INVARIANT_UUID = "b5e3708e-30be-484e-9f9c-112ed0c64cb5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.underlayvpn" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=6172e2ce-72ca-43eb-8727-e5694f364023 EntryTimestamp=2021-08-11T16:14:29.757Z InvocationID=cce9ce37-ee75-482e-9d56-1ceea3075dbc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:14:32.474Z [qtp2045766957-18] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VL UNDERLAYVPN" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "6172e2ce-72ca-43eb-8727-e5694f364023" INVARIANT_UUID = "b5e3708e-30be-484e-9f9c-112ed0c64cb5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.underlayvpn" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=6172e2ce-72ca-43eb-8727-e5694f364023 TargetServiceName=null EntryTimestamp=2021-08-11T16:14:29.757Z InvocationID=cce9ce37-ee75-482e-9d56-1ceea3075dbc ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:14:33.055Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VL UNDERLAYVPN" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "6172e2ce-72ca-43eb-8727-e5694f364023" INVARIANT_UUID = "b5e3708e-30be-484e-9f9c-112ed0c64cb5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.underlayvpn" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=6172e2ce-72ca-43eb-8727-e5694f364023 TargetServiceName=null EntryTimestamp=2021-08-11T16:14:29.757Z InvocationID=cce9ce37-ee75-482e-9d56-1ceea3075dbc ErrorCode=500 lock resource with id e89755b1-7d7c-4816-abbd-344de8081378 2021-08-11T16:14:33.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:14:33.319Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:14:34.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9dff2c81-b1f9-4db5-b832-5f729398dc08 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:14:34.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9dff2c81-b1f9-4db5-b832-5f729398dc08 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@260cc36b 2021-08-11T16:14:34.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9dff2c81-b1f9-4db5-b832-5f729398dc08 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:14:34.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9dff2c81-b1f9-4db5-b832-5f729398dc08 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:14:34.683Z [qtp2045766957-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VL UNDERLAYVPN" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "6172e2ce-72ca-43eb-8727-e5694f364023" INVARIANT_UUID = "b5e3708e-30be-484e-9f9c-112ed0c64cb5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "e89755b1-7d7c-4816-abbd-344de8081378.assettoscatemplate" CURR_ARTIFACT_UUID = "e89755b1-7d7c-4816-abbd-344de8081378.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VlUnderlayvpn-template.yml,0,e89755b1-7d7c-4816-abbd-344de8081378.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=aa47d90d-6aef-4ea8-8806-fca335a0b32b TargetServiceName=null EntryTimestamp=2021-08-11T16:14:29.757Z InvocationID=cce9ce37-ee75-482e-9d56-1ceea3075dbc ErrorCode=500 Starting to create Csar for component VL UNDERLAYVPN 2021-08-11T16:14:34.972Z [qtp2045766957-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VL UNDERLAYVPN" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "6172e2ce-72ca-43eb-8727-e5694f364023" INVARIANT_UUID = "b5e3708e-30be-484e-9f9c-112ed0c64cb5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "e89755b1-7d7c-4816-abbd-344de8081378.assettoscatemplate" CURR_ARTIFACT_UUID = "e89755b1-7d7c-4816-abbd-344de8081378.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VlUnderlayvpn-template.yml,0,e89755b1-7d7c-4816-abbd-344de8081378.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=aa47d90d-6aef-4ea8-8806-fca335a0b32b TargetServiceName=null EntryTimestamp=2021-08-11T16:14:29.757Z InvocationID=cce9ce37-ee75-482e-9d56-1ceea3075dbc ErrorCode=500 Ended create Csar for component VL UNDERLAYVPN 2021-08-11T16:14:34.973Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VL UNDERLAYVPN" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "6172e2ce-72ca-43eb-8727-e5694f364023" INVARIANT_UUID = "b5e3708e-30be-484e-9f9c-112ed0c64cb5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "e89755b1-7d7c-4816-abbd-344de8081378.assettoscatemplate" CURR_ARTIFACT_UUID = "e89755b1-7d7c-4816-abbd-344de8081378.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VlUnderlayvpn-template.yml,0,e89755b1-7d7c-4816-abbd-344de8081378.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=aa47d90d-6aef-4ea8-8806-fca335a0b32b TargetServiceName=null EntryTimestamp=2021-08-11T16:14:29.757Z InvocationID=cce9ce37-ee75-482e-9d56-1ceea3075dbc ErrorCode=500 lock resource with id e89755b1-7d7c-4816-abbd-344de8081378 2021-08-11T16:14:36.767Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VL UNDERLAYVPN" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "6172e2ce-72ca-43eb-8727-e5694f364023" INVARIANT_UUID = "b5e3708e-30be-484e-9f9c-112ed0c64cb5" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "e89755b1-7d7c-4816-abbd-344de8081378.assettoscacsar" CURR_ARTIFACT_UUID = "e89755b1-7d7c-4816-abbd-344de8081378.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VlUnderlayvpn-csar.csar,0,e89755b1-7d7c-4816-abbd-344de8081378.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=7009 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7afb1fc9-da39-45aa-a77b-accda15d63a5 TargetServiceName=null EntryTimestamp=2021-08-11T16:14:29.757Z InvocationID=cce9ce37-ee75-482e-9d56-1ceea3075dbc ErrorCode=500 LogTimestamp=2021-08-11T16:14:36.766945Z ResponseDesc=Created Exiting. 2021-08-11T16:14:36.768Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:14:36.788Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=2836f8e4-05a1-4f65-8fb1-182cc15fd625 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:14:36.788184Z InvocationID=1f10bbab-f49d-4037-807a-871ac2e512a0 LogTimestamp=2021-08-11T16:14:36.788238Z Entering 2021-08-11T16:14:36.788Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=2836f8e4-05a1-4f65-8fb1-182cc15fd625 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:14:36.788184Z InvocationID=1f10bbab-f49d-4037-807a-871ac2e512a0 LogTimestamp=2021-08-11T16:14:36.788238Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:14:37.164Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:14:36.788Z InvocationID=40d65576-5e03-4078-a26a-0cc0762d918a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:14:37.164Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:14:36.788Z InvocationID=40d65576-5e03-4078-a26a-0cc0762d918a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:14:37.164Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:14:36.788Z InvocationID=40d65576-5e03-4078-a26a-0cc0762d918a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:14:37.164Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:14:36.788Z InvocationID=40d65576-5e03-4078-a26a-0cc0762d918a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:14:37.192Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:14:36.788Z InvocationID=40d65576-5e03-4078-a26a-0cc0762d918a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name VlOverlaytunnel 2021-08-11T16:14:37.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0cb3c82d-cd63-481b-99e1-e153c13925b3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:14:37.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0cb3c82d-cd63-481b-99e1-e153c13925b3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@649a3356 2021-08-11T16:14:37.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0cb3c82d-cd63-481b-99e1-e153c13925b3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:14:37.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0cb3c82d-cd63-481b-99e1-e153c13925b3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:14:39.081Z [qtp2045766957-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VL OVERLAYTUNNEL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "af48a29b-472b-412a-84c2-a79eb8eb2523" INVARIANT_UUID = "e1f16395-bf80-4d6f-9183-6174290fb18d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.overlaytunnel" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=af48a29b-472b-412a-84c2-a79eb8eb2523 EntryTimestamp=2021-08-11T16:14:36.788Z InvocationID=40d65576-5e03-4078-a26a-0cc0762d918a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:14:39.082Z [qtp2045766957-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VL OVERLAYTUNNEL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "af48a29b-472b-412a-84c2-a79eb8eb2523" INVARIANT_UUID = "e1f16395-bf80-4d6f-9183-6174290fb18d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.overlaytunnel" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=af48a29b-472b-412a-84c2-a79eb8eb2523 TargetServiceName=null EntryTimestamp=2021-08-11T16:14:36.788Z InvocationID=40d65576-5e03-4078-a26a-0cc0762d918a ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:14:39.672Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VL OVERLAYTUNNEL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "af48a29b-472b-412a-84c2-a79eb8eb2523" INVARIANT_UUID = "e1f16395-bf80-4d6f-9183-6174290fb18d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.overlaytunnel" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=af48a29b-472b-412a-84c2-a79eb8eb2523 TargetServiceName=null EntryTimestamp=2021-08-11T16:14:36.788Z InvocationID=40d65576-5e03-4078-a26a-0cc0762d918a ErrorCode=500 lock resource with id c63fa0a6-faed-4caa-9baa-b8730d429caa 2021-08-11T16:14:40.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=912e5aa9-61c3-4ffa-bd4d-2b0c2cbfcf2a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:14:40.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=912e5aa9-61c3-4ffa-bd4d-2b0c2cbfcf2a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2fcce376 2021-08-11T16:14:40.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=912e5aa9-61c3-4ffa-bd4d-2b0c2cbfcf2a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:14:40.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=912e5aa9-61c3-4ffa-bd4d-2b0c2cbfcf2a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:14:41.551Z [qtp2045766957-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VL OVERLAYTUNNEL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "af48a29b-472b-412a-84c2-a79eb8eb2523" INVARIANT_UUID = "e1f16395-bf80-4d6f-9183-6174290fb18d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c63fa0a6-faed-4caa-9baa-b8730d429caa.assettoscatemplate" CURR_ARTIFACT_UUID = "c63fa0a6-faed-4caa-9baa-b8730d429caa.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VlOverlaytunnel-template.yml,0,c63fa0a6-faed-4caa-9baa-b8730d429caa.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c5c209f9-3a54-4a06-947a-07f79f4ce322 TargetServiceName=null EntryTimestamp=2021-08-11T16:14:36.788Z InvocationID=40d65576-5e03-4078-a26a-0cc0762d918a ErrorCode=500 Starting to create Csar for component VL OVERLAYTUNNEL 2021-08-11T16:14:41.725Z [qtp2045766957-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VL OVERLAYTUNNEL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "af48a29b-472b-412a-84c2-a79eb8eb2523" INVARIANT_UUID = "e1f16395-bf80-4d6f-9183-6174290fb18d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c63fa0a6-faed-4caa-9baa-b8730d429caa.assettoscatemplate" CURR_ARTIFACT_UUID = "c63fa0a6-faed-4caa-9baa-b8730d429caa.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VlOverlaytunnel-template.yml,0,c63fa0a6-faed-4caa-9baa-b8730d429caa.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c5c209f9-3a54-4a06-947a-07f79f4ce322 TargetServiceName=null EntryTimestamp=2021-08-11T16:14:36.788Z InvocationID=40d65576-5e03-4078-a26a-0cc0762d918a ErrorCode=500 Ended create Csar for component VL OVERLAYTUNNEL 2021-08-11T16:14:41.726Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VL OVERLAYTUNNEL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "af48a29b-472b-412a-84c2-a79eb8eb2523" INVARIANT_UUID = "e1f16395-bf80-4d6f-9183-6174290fb18d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c63fa0a6-faed-4caa-9baa-b8730d429caa.assettoscatemplate" CURR_ARTIFACT_UUID = "c63fa0a6-faed-4caa-9baa-b8730d429caa.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VlOverlaytunnel-template.yml,0,c63fa0a6-faed-4caa-9baa-b8730d429caa.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c5c209f9-3a54-4a06-947a-07f79f4ce322 TargetServiceName=null EntryTimestamp=2021-08-11T16:14:36.788Z InvocationID=40d65576-5e03-4078-a26a-0cc0762d918a ErrorCode=500 lock resource with id c63fa0a6-faed-4caa-9baa-b8730d429caa 2021-08-11T16:14:43.392Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VL OVERLAYTUNNEL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "af48a29b-472b-412a-84c2-a79eb8eb2523" INVARIANT_UUID = "e1f16395-bf80-4d6f-9183-6174290fb18d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c63fa0a6-faed-4caa-9baa-b8730d429caa.assettoscacsar" CURR_ARTIFACT_UUID = "c63fa0a6-faed-4caa-9baa-b8730d429caa.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VlOverlaytunnel-csar.csar,0,c63fa0a6-faed-4caa-9baa-b8730d429caa.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=6603 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b2695e0b-89c2-4399-bf8d-aeddb831fdca TargetServiceName=null EntryTimestamp=2021-08-11T16:14:36.788Z InvocationID=40d65576-5e03-4078-a26a-0cc0762d918a ErrorCode=500 LogTimestamp=2021-08-11T16:14:43.391702Z ResponseDesc=Created Exiting. 2021-08-11T16:14:43.392Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:14:43.414Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=73a52f77-a575-424e-9802-7cdac38c1cba InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:14:43.414316Z InvocationID=c7bf9d4d-0b20-4162-b376-71c7a6e04f4c LogTimestamp=2021-08-11T16:14:43.414359Z Entering 2021-08-11T16:14:43.414Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=73a52f77-a575-424e-9802-7cdac38c1cba InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:14:43.414316Z InvocationID=c7bf9d4d-0b20-4162-b376-71c7a6e04f4c LogTimestamp=2021-08-11T16:14:43.414359Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:14:43.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76e164f9-8500-4693-8315-4e7080dd9459 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:14:43.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76e164f9-8500-4693-8315-4e7080dd9459 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@278df0b3 2021-08-11T16:14:43.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76e164f9-8500-4693-8315-4e7080dd9459 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:14:43.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76e164f9-8500-4693-8315-4e7080dd9459 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:14:44.673Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:14:43.415Z InvocationID=96f3c63c-47d2-4600-87fe-e9013fb0aa22 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:14:44.673Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:14:43.415Z InvocationID=96f3c63c-47d2-4600-87fe-e9013fb0aa22 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:14:44.673Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:14:43.415Z InvocationID=96f3c63c-47d2-4600-87fe-e9013fb0aa22 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:14:44.673Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:14:43.415Z InvocationID=96f3c63c-47d2-4600-87fe-e9013fb0aa22 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:14:44.703Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:14:43.415Z InvocationID=96f3c63c-47d2-4600-87fe-e9013fb0aa22 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name GenericNeutronnet 2021-08-11T16:14:46.090Z [qtp2045766957-18] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Generic NeutronNet" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "b0bac2ba-b67b-441c-ada8-9262ce6e5941" INVARIANT_UUID = "57878a4b-d0c4-4fa2-8fc2-546e17e548af" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.GenericNeutronNet" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b0bac2ba-b67b-441c-ada8-9262ce6e5941 EntryTimestamp=2021-08-11T16:14:43.415Z InvocationID=96f3c63c-47d2-4600-87fe-e9013fb0aa22 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:14:46.091Z [qtp2045766957-18] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Generic NeutronNet" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "b0bac2ba-b67b-441c-ada8-9262ce6e5941" INVARIANT_UUID = "57878a4b-d0c4-4fa2-8fc2-546e17e548af" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.GenericNeutronNet" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b0bac2ba-b67b-441c-ada8-9262ce6e5941 TargetServiceName=null EntryTimestamp=2021-08-11T16:14:43.415Z InvocationID=96f3c63c-47d2-4600-87fe-e9013fb0aa22 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:14:46.846Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9cab99f1-5483-4240-9e0e-116b1c95642e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:14:46.846Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9cab99f1-5483-4240-9e0e-116b1c95642e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@675a68b2 2021-08-11T16:14:46.846Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9cab99f1-5483-4240-9e0e-116b1c95642e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:14:46.846Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9cab99f1-5483-4240-9e0e-116b1c95642e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:14:47.174Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Generic NeutronNet" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "b0bac2ba-b67b-441c-ada8-9262ce6e5941" INVARIANT_UUID = "57878a4b-d0c4-4fa2-8fc2-546e17e548af" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vl.GenericNeutronNet" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b0bac2ba-b67b-441c-ada8-9262ce6e5941 TargetServiceName=null EntryTimestamp=2021-08-11T16:14:43.415Z InvocationID=96f3c63c-47d2-4600-87fe-e9013fb0aa22 ErrorCode=500 lock resource with id 749889f3-770b-4d89-975f-6020f78ccb1c 2021-08-11T16:14:48.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:14:48.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:14:48.808Z [qtp2045766957-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Generic NeutronNet" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "b0bac2ba-b67b-441c-ada8-9262ce6e5941" INVARIANT_UUID = "57878a4b-d0c4-4fa2-8fc2-546e17e548af" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "749889f3-770b-4d89-975f-6020f78ccb1c.assettoscatemplate" CURR_ARTIFACT_UUID = "749889f3-770b-4d89-975f-6020f78ccb1c.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-GenericNeutronnet-template.yml,0,749889f3-770b-4d89-975f-6020f78ccb1c.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=dcf3c4bb-65e9-4005-8a5e-4a97aa2f6b3d TargetServiceName=null EntryTimestamp=2021-08-11T16:14:43.415Z InvocationID=96f3c63c-47d2-4600-87fe-e9013fb0aa22 ErrorCode=500 Starting to create Csar for component Generic NeutronNet 2021-08-11T16:14:48.997Z [qtp2045766957-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Generic NeutronNet" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "b0bac2ba-b67b-441c-ada8-9262ce6e5941" INVARIANT_UUID = "57878a4b-d0c4-4fa2-8fc2-546e17e548af" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "749889f3-770b-4d89-975f-6020f78ccb1c.assettoscatemplate" CURR_ARTIFACT_UUID = "749889f3-770b-4d89-975f-6020f78ccb1c.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-GenericNeutronnet-template.yml,0,749889f3-770b-4d89-975f-6020f78ccb1c.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=dcf3c4bb-65e9-4005-8a5e-4a97aa2f6b3d TargetServiceName=null EntryTimestamp=2021-08-11T16:14:43.415Z InvocationID=96f3c63c-47d2-4600-87fe-e9013fb0aa22 ErrorCode=500 Ended create Csar for component Generic NeutronNet 2021-08-11T16:14:48.998Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Generic NeutronNet" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "b0bac2ba-b67b-441c-ada8-9262ce6e5941" INVARIANT_UUID = "57878a4b-d0c4-4fa2-8fc2-546e17e548af" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "749889f3-770b-4d89-975f-6020f78ccb1c.assettoscatemplate" CURR_ARTIFACT_UUID = "749889f3-770b-4d89-975f-6020f78ccb1c.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-GenericNeutronnet-template.yml,0,749889f3-770b-4d89-975f-6020f78ccb1c.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=dcf3c4bb-65e9-4005-8a5e-4a97aa2f6b3d TargetServiceName=null EntryTimestamp=2021-08-11T16:14:43.415Z InvocationID=96f3c63c-47d2-4600-87fe-e9013fb0aa22 ErrorCode=500 lock resource with id 749889f3-770b-4d89-975f-6020f78ccb1c 2021-08-11T16:14:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:14:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:14:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:14:49.731Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec33f789-489e-4d52-8ffe-f7de55bcac8e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:14:49.731Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec33f789-489e-4d52-8ffe-f7de55bcac8e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@52efe78c 2021-08-11T16:14:49.731Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec33f789-489e-4d52-8ffe-f7de55bcac8e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:14:49.731Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec33f789-489e-4d52-8ffe-f7de55bcac8e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:14:50.762Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Generic NeutronNet" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "b0bac2ba-b67b-441c-ada8-9262ce6e5941" INVARIANT_UUID = "57878a4b-d0c4-4fa2-8fc2-546e17e548af" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "749889f3-770b-4d89-975f-6020f78ccb1c.assettoscacsar" CURR_ARTIFACT_UUID = "749889f3-770b-4d89-975f-6020f78ccb1c.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-GenericNeutronnet-csar.csar,0,749889f3-770b-4d89-975f-6020f78ccb1c.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=7347 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ef4ad76b-44bd-4c3f-8f8d-c377231a08a9 TargetServiceName=null EntryTimestamp=2021-08-11T16:14:43.415Z InvocationID=96f3c63c-47d2-4600-87fe-e9013fb0aa22 ErrorCode=500 LogTimestamp=2021-08-11T16:14:50.762068Z ResponseDesc=Created Exiting. 2021-08-11T16:14:50.764Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:14:50.786Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=54202a6a-1815-45e3-8a87-f8d80d60a555 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:14:50.785858Z InvocationID=587e414a-7614-46e9-ac60-ab8c6d4c99de LogTimestamp=2021-08-11T16:14:50.785911Z Entering 2021-08-11T16:14:50.786Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=54202a6a-1815-45e3-8a87-f8d80d60a555 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:14:50.785858Z InvocationID=587e414a-7614-46e9-ac60-ab8c6d4c99de LogTimestamp=2021-08-11T16:14:50.785911Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:14:51.569Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:14:50.786Z InvocationID=d48db97b-ccd9-49f8-adf9-5343d45cbd02 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:14:51.569Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:14:50.786Z InvocationID=d48db97b-ccd9-49f8-adf9-5343d45cbd02 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:14:51.569Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:14:50.786Z InvocationID=d48db97b-ccd9-49f8-adf9-5343d45cbd02 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:14:51.569Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:14:50.786Z InvocationID=d48db97b-ccd9-49f8-adf9-5343d45cbd02 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:14:51.584Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:14:50.786Z InvocationID=d48db97b-ccd9-49f8-adf9-5343d45cbd02 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Allottedresource 2021-08-11T16:14:52.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1bf11fd-0b38-4bf0-95fd-a6913193f278 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:14:52.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1bf11fd-0b38-4bf0-95fd-a6913193f278 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1a7079f4 2021-08-11T16:14:52.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1bf11fd-0b38-4bf0-95fd-a6913193f278 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:14:52.779Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1bf11fd-0b38-4bf0-95fd-a6913193f278 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:14:53.291Z [qtp2045766957-12] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "AllottedResource" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "1172939b-86c4-41a2-a490-b1136d8c581f" INVARIANT_UUID = "cfbc5590-e560-4dd3-b7a7-ad3f38415357" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.AllottedResource" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1172939b-86c4-41a2-a490-b1136d8c581f EntryTimestamp=2021-08-11T16:14:50.786Z InvocationID=d48db97b-ccd9-49f8-adf9-5343d45cbd02 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:14:53.292Z [qtp2045766957-12] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "AllottedResource" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "1172939b-86c4-41a2-a490-b1136d8c581f" INVARIANT_UUID = "cfbc5590-e560-4dd3-b7a7-ad3f38415357" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.AllottedResource" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1172939b-86c4-41a2-a490-b1136d8c581f TargetServiceName=null EntryTimestamp=2021-08-11T16:14:50.786Z InvocationID=d48db97b-ccd9-49f8-adf9-5343d45cbd02 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:14:53.698Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "AllottedResource" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "1172939b-86c4-41a2-a490-b1136d8c581f" INVARIANT_UUID = "cfbc5590-e560-4dd3-b7a7-ad3f38415357" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.AllottedResource" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1172939b-86c4-41a2-a490-b1136d8c581f TargetServiceName=null EntryTimestamp=2021-08-11T16:14:50.786Z InvocationID=d48db97b-ccd9-49f8-adf9-5343d45cbd02 ErrorCode=500 lock resource with id caaaf4be-e4b7-4856-87e6-3bd6d030081e 2021-08-11T16:14:55.450Z [qtp2045766957-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "AllottedResource" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "1172939b-86c4-41a2-a490-b1136d8c581f" INVARIANT_UUID = "cfbc5590-e560-4dd3-b7a7-ad3f38415357" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "caaaf4be-e4b7-4856-87e6-3bd6d030081e.assettoscatemplate" CURR_ARTIFACT_UUID = "caaaf4be-e4b7-4856-87e6-3bd6d030081e.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Allottedresource-template.yml,0,caaaf4be-e4b7-4856-87e6-3bd6d030081e.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b9658c54-d69d-4848-9d7f-dc453b5a5034 TargetServiceName=null EntryTimestamp=2021-08-11T16:14:50.786Z InvocationID=d48db97b-ccd9-49f8-adf9-5343d45cbd02 ErrorCode=500 Starting to create Csar for component AllottedResource 2021-08-11T16:14:55.625Z [qtp2045766957-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "AllottedResource" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "1172939b-86c4-41a2-a490-b1136d8c581f" INVARIANT_UUID = "cfbc5590-e560-4dd3-b7a7-ad3f38415357" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "caaaf4be-e4b7-4856-87e6-3bd6d030081e.assettoscatemplate" CURR_ARTIFACT_UUID = "caaaf4be-e4b7-4856-87e6-3bd6d030081e.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Allottedresource-template.yml,0,caaaf4be-e4b7-4856-87e6-3bd6d030081e.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b9658c54-d69d-4848-9d7f-dc453b5a5034 TargetServiceName=null EntryTimestamp=2021-08-11T16:14:50.786Z InvocationID=d48db97b-ccd9-49f8-adf9-5343d45cbd02 ErrorCode=500 Ended create Csar for component AllottedResource 2021-08-11T16:14:55.625Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "AllottedResource" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "1172939b-86c4-41a2-a490-b1136d8c581f" INVARIANT_UUID = "cfbc5590-e560-4dd3-b7a7-ad3f38415357" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "caaaf4be-e4b7-4856-87e6-3bd6d030081e.assettoscatemplate" CURR_ARTIFACT_UUID = "caaaf4be-e4b7-4856-87e6-3bd6d030081e.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Allottedresource-template.yml,0,caaaf4be-e4b7-4856-87e6-3bd6d030081e.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b9658c54-d69d-4848-9d7f-dc453b5a5034 TargetServiceName=null EntryTimestamp=2021-08-11T16:14:50.786Z InvocationID=d48db97b-ccd9-49f8-adf9-5343d45cbd02 ErrorCode=500 lock resource with id caaaf4be-e4b7-4856-87e6-3bd6d030081e 2021-08-11T16:14:55.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4885960-9fbb-4774-96a5-3712827737f7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:14:55.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4885960-9fbb-4774-96a5-3712827737f7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@52b00a61 2021-08-11T16:14:55.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4885960-9fbb-4774-96a5-3712827737f7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:14:55.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4885960-9fbb-4774-96a5-3712827737f7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:14:57.280Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "AllottedResource" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "1172939b-86c4-41a2-a490-b1136d8c581f" INVARIANT_UUID = "cfbc5590-e560-4dd3-b7a7-ad3f38415357" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "caaaf4be-e4b7-4856-87e6-3bd6d030081e.assettoscacsar" CURR_ARTIFACT_UUID = "caaaf4be-e4b7-4856-87e6-3bd6d030081e.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Allottedresource-csar.csar,0,caaaf4be-e4b7-4856-87e6-3bd6d030081e.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=6493 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=10569074-0c87-446c-be5b-4f9beceab490 TargetServiceName=null EntryTimestamp=2021-08-11T16:14:50.786Z InvocationID=d48db97b-ccd9-49f8-adf9-5343d45cbd02 ErrorCode=500 LogTimestamp=2021-08-11T16:14:57.279680Z ResponseDesc=Created Exiting. 2021-08-11T16:14:57.281Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:14:57.314Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=85037b79-34ea-4a24-8f6e-56ddf6887626 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:14:57.313734Z InvocationID=b62ed865-6889-469d-ba18-1b633f30c1ee LogTimestamp=2021-08-11T16:14:57.313803Z Entering 2021-08-11T16:14:57.314Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=85037b79-34ea-4a24-8f6e-56ddf6887626 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:14:57.313734Z InvocationID=b62ed865-6889-469d-ba18-1b633f30c1ee LogTimestamp=2021-08-11T16:14:57.313803Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:14:57.655Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:14:57.314Z InvocationID=1e14bd51-58d6-4ded-8bd6-88b7ecf2064f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:14:57.655Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:14:57.314Z InvocationID=1e14bd51-58d6-4ded-8bd6-88b7ecf2064f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:14:57.655Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:14:57.314Z InvocationID=1e14bd51-58d6-4ded-8bd6-88b7ecf2064f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:14:57.655Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:14:57.314Z InvocationID=1e14bd51-58d6-4ded-8bd6-88b7ecf2064f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:14:57.678Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:14:57.314Z InvocationID=1e14bd51-58d6-4ded-8bd6-88b7ecf2064f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name ExtImageFile 2021-08-11T16:14:58.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a5055e4e-6855-48f8-9417-f42970237b43 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:14:58.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a5055e4e-6855-48f8-9417-f42970237b43 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@661be77d 2021-08-11T16:14:58.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a5055e4e-6855-48f8-9417-f42970237b43 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:14:58.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a5055e4e-6855-48f8-9417-f42970237b43 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:14:59.512Z [qtp2045766957-18] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Ext Image File" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "d1f59863-44fe-489c-a66d-0dff6299dc54" INVARIANT_UUID = "a1d1a7a4-0d1a-4b57-bb9d-77fbe0aa5547" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.ext.ImageFile" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d1f59863-44fe-489c-a66d-0dff6299dc54 EntryTimestamp=2021-08-11T16:14:57.314Z InvocationID=1e14bd51-58d6-4ded-8bd6-88b7ecf2064f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:14:59.513Z [qtp2045766957-18] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Ext Image File" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "d1f59863-44fe-489c-a66d-0dff6299dc54" INVARIANT_UUID = "a1d1a7a4-0d1a-4b57-bb9d-77fbe0aa5547" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.ext.ImageFile" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d1f59863-44fe-489c-a66d-0dff6299dc54 TargetServiceName=null EntryTimestamp=2021-08-11T16:14:57.314Z InvocationID=1e14bd51-58d6-4ded-8bd6-88b7ecf2064f ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:14:59.909Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Ext Image File" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "d1f59863-44fe-489c-a66d-0dff6299dc54" INVARIANT_UUID = "a1d1a7a4-0d1a-4b57-bb9d-77fbe0aa5547" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.ext.ImageFile" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d1f59863-44fe-489c-a66d-0dff6299dc54 TargetServiceName=null EntryTimestamp=2021-08-11T16:14:57.314Z InvocationID=1e14bd51-58d6-4ded-8bd6-88b7ecf2064f ErrorCode=500 lock resource with id 0da125eb-285a-46ff-80d1-12bd9019f0e1 2021-08-11T16:15:01.573Z [qtp2045766957-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Ext Image File" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "d1f59863-44fe-489c-a66d-0dff6299dc54" INVARIANT_UUID = "a1d1a7a4-0d1a-4b57-bb9d-77fbe0aa5547" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "0da125eb-285a-46ff-80d1-12bd9019f0e1.assettoscatemplate" CURR_ARTIFACT_UUID = "0da125eb-285a-46ff-80d1-12bd9019f0e1.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-ExtImageFile-template.yml,0,0da125eb-285a-46ff-80d1-12bd9019f0e1.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=774384a7-0c9a-4312-9006-941d15863947 TargetServiceName=null EntryTimestamp=2021-08-11T16:14:57.314Z InvocationID=1e14bd51-58d6-4ded-8bd6-88b7ecf2064f ErrorCode=500 Starting to create Csar for component Ext Image File 2021-08-11T16:15:01.753Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d76de21-eb15-4d5a-8489-5d110f744714 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:15:01.754Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d76de21-eb15-4d5a-8489-5d110f744714 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4b65afb5 2021-08-11T16:15:01.754Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d76de21-eb15-4d5a-8489-5d110f744714 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:15:01.754Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d76de21-eb15-4d5a-8489-5d110f744714 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:15:01.784Z [qtp2045766957-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Ext Image File" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "d1f59863-44fe-489c-a66d-0dff6299dc54" INVARIANT_UUID = "a1d1a7a4-0d1a-4b57-bb9d-77fbe0aa5547" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "0da125eb-285a-46ff-80d1-12bd9019f0e1.assettoscatemplate" CURR_ARTIFACT_UUID = "0da125eb-285a-46ff-80d1-12bd9019f0e1.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-ExtImageFile-template.yml,0,0da125eb-285a-46ff-80d1-12bd9019f0e1.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=774384a7-0c9a-4312-9006-941d15863947 TargetServiceName=null EntryTimestamp=2021-08-11T16:14:57.314Z InvocationID=1e14bd51-58d6-4ded-8bd6-88b7ecf2064f ErrorCode=500 Ended create Csar for component Ext Image File 2021-08-11T16:15:01.784Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Ext Image File" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "d1f59863-44fe-489c-a66d-0dff6299dc54" INVARIANT_UUID = "a1d1a7a4-0d1a-4b57-bb9d-77fbe0aa5547" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "0da125eb-285a-46ff-80d1-12bd9019f0e1.assettoscatemplate" CURR_ARTIFACT_UUID = "0da125eb-285a-46ff-80d1-12bd9019f0e1.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-ExtImageFile-template.yml,0,0da125eb-285a-46ff-80d1-12bd9019f0e1.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=774384a7-0c9a-4312-9006-941d15863947 TargetServiceName=null EntryTimestamp=2021-08-11T16:14:57.314Z InvocationID=1e14bd51-58d6-4ded-8bd6-88b7ecf2064f ErrorCode=500 lock resource with id 0da125eb-285a-46ff-80d1-12bd9019f0e1 2021-08-11T16:15:03.309Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:15:03.318Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:15:03.436Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Ext Image File" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "d1f59863-44fe-489c-a66d-0dff6299dc54" INVARIANT_UUID = "a1d1a7a4-0d1a-4b57-bb9d-77fbe0aa5547" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "0da125eb-285a-46ff-80d1-12bd9019f0e1.assettoscacsar" CURR_ARTIFACT_UUID = "0da125eb-285a-46ff-80d1-12bd9019f0e1.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-ExtImageFile-csar.csar,0,0da125eb-285a-46ff-80d1-12bd9019f0e1.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=6121 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=754082eb-4894-4a0d-ac51-14097d7d49b8 TargetServiceName=null EntryTimestamp=2021-08-11T16:14:57.314Z InvocationID=1e14bd51-58d6-4ded-8bd6-88b7ecf2064f ErrorCode=500 LogTimestamp=2021-08-11T16:15:03.435908Z ResponseDesc=Created Exiting. 2021-08-11T16:15:03.436Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:15:03.458Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=ff8f4713-8b26-4e16-a0d5-09f550948eb9 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:15:03.458086Z InvocationID=8c3e2629-6ee8-495d-9989-2e86027ff03c LogTimestamp=2021-08-11T16:15:03.458147Z Entering 2021-08-11T16:15:03.458Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=ff8f4713-8b26-4e16-a0d5-09f550948eb9 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:15:03.458086Z InvocationID=8c3e2629-6ee8-495d-9989-2e86027ff03c LogTimestamp=2021-08-11T16:15:03.458147Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:15:03.841Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:03.459Z InvocationID=8dbfdab8-3cbe-4cf5-b617-57fb2c49e85c PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:15:03.841Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:03.459Z InvocationID=8dbfdab8-3cbe-4cf5-b617-57fb2c49e85c PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:15:03.841Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:03.459Z InvocationID=8dbfdab8-3cbe-4cf5-b617-57fb2c49e85c PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:15:03.841Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:03.459Z InvocationID=8dbfdab8-3cbe-4cf5-b617-57fb2c49e85c PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:15:03.856Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:03.459Z InvocationID=8dbfdab8-3cbe-4cf5-b617-57fb2c49e85c PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name ExtLocalStorage 2021-08-11T16:15:04.299Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:15:04.995Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=faf1206c-a834-4054-8387-b5f2752d17d4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:15:04.995Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=faf1206c-a834-4054-8387-b5f2752d17d4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@52c3c0f4 2021-08-11T16:15:04.995Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=faf1206c-a834-4054-8387-b5f2752d17d4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:15:04.995Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=faf1206c-a834-4054-8387-b5f2752d17d4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:15:05.495Z [qtp2045766957-14] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Ext Local Storage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "1675bc4b-8077-439e-b6fd-a150f6dc0003" INVARIANT_UUID = "e655eafd-0476-410b-abd4-e9ab74b8e374" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.ext.LocalStorage" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1675bc4b-8077-439e-b6fd-a150f6dc0003 EntryTimestamp=2021-08-11T16:15:03.459Z InvocationID=8dbfdab8-3cbe-4cf5-b617-57fb2c49e85c PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:15:05.496Z [qtp2045766957-14] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Ext Local Storage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "1675bc4b-8077-439e-b6fd-a150f6dc0003" INVARIANT_UUID = "e655eafd-0476-410b-abd4-e9ab74b8e374" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.ext.LocalStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1675bc4b-8077-439e-b6fd-a150f6dc0003 TargetServiceName=null EntryTimestamp=2021-08-11T16:15:03.459Z InvocationID=8dbfdab8-3cbe-4cf5-b617-57fb2c49e85c ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:15:05.927Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Ext Local Storage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "1675bc4b-8077-439e-b6fd-a150f6dc0003" INVARIANT_UUID = "e655eafd-0476-410b-abd4-e9ab74b8e374" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.ext.LocalStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1675bc4b-8077-439e-b6fd-a150f6dc0003 TargetServiceName=null EntryTimestamp=2021-08-11T16:15:03.459Z InvocationID=8dbfdab8-3cbe-4cf5-b617-57fb2c49e85c ErrorCode=500 lock resource with id e339c50b-2bba-46ab-baaa-7311e849b91f 2021-08-11T16:15:07.584Z [qtp2045766957-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Ext Local Storage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "1675bc4b-8077-439e-b6fd-a150f6dc0003" INVARIANT_UUID = "e655eafd-0476-410b-abd4-e9ab74b8e374" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "e339c50b-2bba-46ab-baaa-7311e849b91f.assettoscatemplate" CURR_ARTIFACT_UUID = "e339c50b-2bba-46ab-baaa-7311e849b91f.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-ExtLocalStorage-template.yml,0,e339c50b-2bba-46ab-baaa-7311e849b91f.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=73f23d6f-40f7-4819-a513-f2422a1bb72c TargetServiceName=null EntryTimestamp=2021-08-11T16:15:03.459Z InvocationID=8dbfdab8-3cbe-4cf5-b617-57fb2c49e85c ErrorCode=500 Starting to create Csar for component Ext Local Storage 2021-08-11T16:15:07.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4dd976b9-7d1f-42f7-bb3f-6225ed71ecf5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:15:07.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4dd976b9-7d1f-42f7-bb3f-6225ed71ecf5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@f22bcf 2021-08-11T16:15:07.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4dd976b9-7d1f-42f7-bb3f-6225ed71ecf5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:15:07.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4dd976b9-7d1f-42f7-bb3f-6225ed71ecf5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:15:07.784Z [qtp2045766957-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Ext Local Storage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "1675bc4b-8077-439e-b6fd-a150f6dc0003" INVARIANT_UUID = "e655eafd-0476-410b-abd4-e9ab74b8e374" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "e339c50b-2bba-46ab-baaa-7311e849b91f.assettoscatemplate" CURR_ARTIFACT_UUID = "e339c50b-2bba-46ab-baaa-7311e849b91f.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-ExtLocalStorage-template.yml,0,e339c50b-2bba-46ab-baaa-7311e849b91f.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=73f23d6f-40f7-4819-a513-f2422a1bb72c TargetServiceName=null EntryTimestamp=2021-08-11T16:15:03.459Z InvocationID=8dbfdab8-3cbe-4cf5-b617-57fb2c49e85c ErrorCode=500 Ended create Csar for component Ext Local Storage 2021-08-11T16:15:07.784Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Ext Local Storage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "1675bc4b-8077-439e-b6fd-a150f6dc0003" INVARIANT_UUID = "e655eafd-0476-410b-abd4-e9ab74b8e374" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "e339c50b-2bba-46ab-baaa-7311e849b91f.assettoscatemplate" CURR_ARTIFACT_UUID = "e339c50b-2bba-46ab-baaa-7311e849b91f.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-ExtLocalStorage-template.yml,0,e339c50b-2bba-46ab-baaa-7311e849b91f.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=73f23d6f-40f7-4819-a513-f2422a1bb72c TargetServiceName=null EntryTimestamp=2021-08-11T16:15:03.459Z InvocationID=8dbfdab8-3cbe-4cf5-b617-57fb2c49e85c ErrorCode=500 lock resource with id e339c50b-2bba-46ab-baaa-7311e849b91f 2021-08-11T16:15:09.429Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Ext Local Storage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "1675bc4b-8077-439e-b6fd-a150f6dc0003" INVARIANT_UUID = "e655eafd-0476-410b-abd4-e9ab74b8e374" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "e339c50b-2bba-46ab-baaa-7311e849b91f.assettoscacsar" CURR_ARTIFACT_UUID = "e339c50b-2bba-46ab-baaa-7311e849b91f.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-ExtLocalStorage-csar.csar,0,e339c50b-2bba-46ab-baaa-7311e849b91f.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5970 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d600d670-4836-4d82-b785-7ad2c2e9ba8b TargetServiceName=null EntryTimestamp=2021-08-11T16:15:03.459Z InvocationID=8dbfdab8-3cbe-4cf5-b617-57fb2c49e85c ErrorCode=500 LogTimestamp=2021-08-11T16:15:09.429356Z ResponseDesc=Created Exiting. 2021-08-11T16:15:09.430Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:15:09.450Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=c9a17852-a905-4b0f-8408-c70048251446 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:15:09.450511Z InvocationID=42f63f0d-d9b7-416a-aae0-54503d0fe06b LogTimestamp=2021-08-11T16:15:09.450631Z Entering 2021-08-11T16:15:09.451Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=c9a17852-a905-4b0f-8408-c70048251446 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:15:09.450511Z InvocationID=42f63f0d-d9b7-416a-aae0-54503d0fe06b LogTimestamp=2021-08-11T16:15:09.450631Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:15:09.984Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:09.451Z InvocationID=282f3604-cdad-44be-8c19-96e825700509 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:15:09.984Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:09.451Z InvocationID=282f3604-cdad-44be-8c19-96e825700509 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:15:09.984Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:09.451Z InvocationID=282f3604-cdad-44be-8c19-96e825700509 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:15:09.984Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:09.451Z InvocationID=282f3604-cdad-44be-8c19-96e825700509 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:15:10.089Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:09.451Z InvocationID=282f3604-cdad-44be-8c19-96e825700509 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name ExtZteCp 2021-08-11T16:15:10.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e441015-a6c5-4bc0-a498-94a1ca4df13b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:15:10.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e441015-a6c5-4bc0-a498-94a1ca4df13b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@131da34a 2021-08-11T16:15:10.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e441015-a6c5-4bc0-a498-94a1ca4df13b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:15:10.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e441015-a6c5-4bc0-a498-94a1ca4df13b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:15:12.082Z [qtp2045766957-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ext ZTE CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "b7ef65c8-9c4b-4457-8d2e-9f82e8a9ab5b" INVARIANT_UUID = "f5fba2b3-d8b6-4b41-ae96-b363ef21c694" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.ext.zte.CP" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b7ef65c8-9c4b-4457-8d2e-9f82e8a9ab5b EntryTimestamp=2021-08-11T16:15:09.451Z InvocationID=282f3604-cdad-44be-8c19-96e825700509 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:15:12.083Z [qtp2045766957-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ext ZTE CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "b7ef65c8-9c4b-4457-8d2e-9f82e8a9ab5b" INVARIANT_UUID = "f5fba2b3-d8b6-4b41-ae96-b363ef21c694" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.ext.zte.CP" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b7ef65c8-9c4b-4457-8d2e-9f82e8a9ab5b TargetServiceName=null EntryTimestamp=2021-08-11T16:15:09.451Z InvocationID=282f3604-cdad-44be-8c19-96e825700509 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:15:12.591Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ext ZTE CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "b7ef65c8-9c4b-4457-8d2e-9f82e8a9ab5b" INVARIANT_UUID = "f5fba2b3-d8b6-4b41-ae96-b363ef21c694" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.ext.zte.CP" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b7ef65c8-9c4b-4457-8d2e-9f82e8a9ab5b TargetServiceName=null EntryTimestamp=2021-08-11T16:15:09.451Z InvocationID=282f3604-cdad-44be-8c19-96e825700509 ErrorCode=500 lock resource with id 8fb74464-114d-4547-aed2-96739ecf1cf3 2021-08-11T16:15:13.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31714e82-5626-4d52-ad10-1a2ae7a7f317 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:15:13.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31714e82-5626-4d52-ad10-1a2ae7a7f317 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2a04eb5 2021-08-11T16:15:13.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31714e82-5626-4d52-ad10-1a2ae7a7f317 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:15:13.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31714e82-5626-4d52-ad10-1a2ae7a7f317 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:15:14.293Z [qtp2045766957-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ext ZTE CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "b7ef65c8-9c4b-4457-8d2e-9f82e8a9ab5b" INVARIANT_UUID = "f5fba2b3-d8b6-4b41-ae96-b363ef21c694" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "8fb74464-114d-4547-aed2-96739ecf1cf3.assettoscatemplate" CURR_ARTIFACT_UUID = "8fb74464-114d-4547-aed2-96739ecf1cf3.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-ExtZteCp-template.yml,0,8fb74464-114d-4547-aed2-96739ecf1cf3.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3bb815ba-d735-4a62-b4c3-1907f1dabe2f TargetServiceName=null EntryTimestamp=2021-08-11T16:15:09.451Z InvocationID=282f3604-cdad-44be-8c19-96e825700509 ErrorCode=500 Starting to create Csar for component ext ZTE CP 2021-08-11T16:15:14.529Z [qtp2045766957-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ext ZTE CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "b7ef65c8-9c4b-4457-8d2e-9f82e8a9ab5b" INVARIANT_UUID = "f5fba2b3-d8b6-4b41-ae96-b363ef21c694" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "8fb74464-114d-4547-aed2-96739ecf1cf3.assettoscatemplate" CURR_ARTIFACT_UUID = "8fb74464-114d-4547-aed2-96739ecf1cf3.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-ExtZteCp-template.yml,0,8fb74464-114d-4547-aed2-96739ecf1cf3.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3bb815ba-d735-4a62-b4c3-1907f1dabe2f TargetServiceName=null EntryTimestamp=2021-08-11T16:15:09.451Z InvocationID=282f3604-cdad-44be-8c19-96e825700509 ErrorCode=500 Ended create Csar for component ext ZTE CP 2021-08-11T16:15:14.530Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ext ZTE CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "b7ef65c8-9c4b-4457-8d2e-9f82e8a9ab5b" INVARIANT_UUID = "f5fba2b3-d8b6-4b41-ae96-b363ef21c694" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "8fb74464-114d-4547-aed2-96739ecf1cf3.assettoscatemplate" CURR_ARTIFACT_UUID = "8fb74464-114d-4547-aed2-96739ecf1cf3.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-ExtZteCp-template.yml,0,8fb74464-114d-4547-aed2-96739ecf1cf3.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3bb815ba-d735-4a62-b4c3-1907f1dabe2f TargetServiceName=null EntryTimestamp=2021-08-11T16:15:09.451Z InvocationID=282f3604-cdad-44be-8c19-96e825700509 ErrorCode=500 lock resource with id 8fb74464-114d-4547-aed2-96739ecf1cf3 2021-08-11T16:15:16.199Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ext ZTE CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "b7ef65c8-9c4b-4457-8d2e-9f82e8a9ab5b" INVARIANT_UUID = "f5fba2b3-d8b6-4b41-ae96-b363ef21c694" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "8fb74464-114d-4547-aed2-96739ecf1cf3.assettoscacsar" CURR_ARTIFACT_UUID = "8fb74464-114d-4547-aed2-96739ecf1cf3.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-ExtZteCp-csar.csar,0,8fb74464-114d-4547-aed2-96739ecf1cf3.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=6747 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=cbc34972-dd6e-4984-9d3c-d2cebeab71af TargetServiceName=null EntryTimestamp=2021-08-11T16:15:09.451Z InvocationID=282f3604-cdad-44be-8c19-96e825700509 ErrorCode=500 LogTimestamp=2021-08-11T16:15:16.198664Z ResponseDesc=Created Exiting. 2021-08-11T16:15:16.199Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:15:16.216Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=f1e2bfbc-fe5f-4749-aa66-664960913835 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:15:16.215897Z InvocationID=0c856235-f24e-4895-bfdf-8b2ade33d42a LogTimestamp=2021-08-11T16:15:16.215944Z Entering 2021-08-11T16:15:16.216Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=f1e2bfbc-fe5f-4749-aa66-664960913835 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:15:16.215897Z InvocationID=0c856235-f24e-4895-bfdf-8b2ade33d42a LogTimestamp=2021-08-11T16:15:16.215944Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:15:16.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1a6a018-f622-43dd-b419-0d7562897be4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:15:16.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1a6a018-f622-43dd-b419-0d7562897be4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@630f1c4e 2021-08-11T16:15:16.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1a6a018-f622-43dd-b419-0d7562897be4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:15:16.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1a6a018-f622-43dd-b419-0d7562897be4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:15:16.959Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:16.216Z InvocationID=b6e4f30a-fd02-4751-afa6-57e5ce8da7ea PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:15:16.959Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:16.216Z InvocationID=b6e4f30a-fd02-4751-afa6-57e5ce8da7ea PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:15:16.959Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:16.216Z InvocationID=b6e4f30a-fd02-4751-afa6-57e5ce8da7ea PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:15:16.959Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:16.216Z InvocationID=b6e4f30a-fd02-4751-afa6-57e5ce8da7ea PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:15:17.105Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:16.216Z InvocationID=b6e4f30a-fd02-4751-afa6-57e5ce8da7ea PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name ExtZteVdu 2021-08-11T16:15:18.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:15:18.318Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:15:18.486Z [qtp2045766957-18] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Ext Zte VDU" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "569f7614-0282-4cde-bbbe-629848a45f8c" INVARIANT_UUID = "78c9c34d-a025-4f1b-89ad-1dcdb041609f" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.ext.zte.VDU" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=569f7614-0282-4cde-bbbe-629848a45f8c EntryTimestamp=2021-08-11T16:15:16.216Z InvocationID=b6e4f30a-fd02-4751-afa6-57e5ce8da7ea PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:15:18.487Z [qtp2045766957-18] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Ext Zte VDU" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "569f7614-0282-4cde-bbbe-629848a45f8c" INVARIANT_UUID = "78c9c34d-a025-4f1b-89ad-1dcdb041609f" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.ext.zte.VDU" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=569f7614-0282-4cde-bbbe-629848a45f8c TargetServiceName=null EntryTimestamp=2021-08-11T16:15:16.216Z InvocationID=b6e4f30a-fd02-4751-afa6-57e5ce8da7ea ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:15:18.865Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Ext Zte VDU" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "569f7614-0282-4cde-bbbe-629848a45f8c" INVARIANT_UUID = "78c9c34d-a025-4f1b-89ad-1dcdb041609f" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.ext.zte.VDU" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=569f7614-0282-4cde-bbbe-629848a45f8c TargetServiceName=null EntryTimestamp=2021-08-11T16:15:16.216Z InvocationID=b6e4f30a-fd02-4751-afa6-57e5ce8da7ea ErrorCode=500 lock resource with id c14beae7-d7f4-4a78-9bee-61a3ec1ee416 2021-08-11T16:15:19.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f983798-4699-486e-a7d9-aca59caaf980 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:15:19.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f983798-4699-486e-a7d9-aca59caaf980 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@140c187f 2021-08-11T16:15:19.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f983798-4699-486e-a7d9-aca59caaf980 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:15:19.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f983798-4699-486e-a7d9-aca59caaf980 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:15:20.584Z [qtp2045766957-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Ext Zte VDU" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "569f7614-0282-4cde-bbbe-629848a45f8c" INVARIANT_UUID = "78c9c34d-a025-4f1b-89ad-1dcdb041609f" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c14beae7-d7f4-4a78-9bee-61a3ec1ee416.assettoscatemplate" CURR_ARTIFACT_UUID = "c14beae7-d7f4-4a78-9bee-61a3ec1ee416.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-ExtZteVdu-template.yml,0,c14beae7-d7f4-4a78-9bee-61a3ec1ee416.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2004aeca-4408-42a6-8988-5001de89ef95 TargetServiceName=null EntryTimestamp=2021-08-11T16:15:16.216Z InvocationID=b6e4f30a-fd02-4751-afa6-57e5ce8da7ea ErrorCode=500 Starting to create Csar for component Ext Zte VDU 2021-08-11T16:15:20.809Z [qtp2045766957-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Ext Zte VDU" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "569f7614-0282-4cde-bbbe-629848a45f8c" INVARIANT_UUID = "78c9c34d-a025-4f1b-89ad-1dcdb041609f" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c14beae7-d7f4-4a78-9bee-61a3ec1ee416.assettoscatemplate" CURR_ARTIFACT_UUID = "c14beae7-d7f4-4a78-9bee-61a3ec1ee416.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-ExtZteVdu-template.yml,0,c14beae7-d7f4-4a78-9bee-61a3ec1ee416.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2004aeca-4408-42a6-8988-5001de89ef95 TargetServiceName=null EntryTimestamp=2021-08-11T16:15:16.216Z InvocationID=b6e4f30a-fd02-4751-afa6-57e5ce8da7ea ErrorCode=500 Ended create Csar for component Ext Zte VDU 2021-08-11T16:15:20.810Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Ext Zte VDU" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "569f7614-0282-4cde-bbbe-629848a45f8c" INVARIANT_UUID = "78c9c34d-a025-4f1b-89ad-1dcdb041609f" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c14beae7-d7f4-4a78-9bee-61a3ec1ee416.assettoscatemplate" CURR_ARTIFACT_UUID = "c14beae7-d7f4-4a78-9bee-61a3ec1ee416.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-ExtZteVdu-template.yml,0,c14beae7-d7f4-4a78-9bee-61a3ec1ee416.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2004aeca-4408-42a6-8988-5001de89ef95 TargetServiceName=null EntryTimestamp=2021-08-11T16:15:16.216Z InvocationID=b6e4f30a-fd02-4751-afa6-57e5ce8da7ea ErrorCode=500 lock resource with id c14beae7-d7f4-4a78-9bee-61a3ec1ee416 2021-08-11T16:15:22.451Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Ext Zte VDU" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "569f7614-0282-4cde-bbbe-629848a45f8c" INVARIANT_UUID = "78c9c34d-a025-4f1b-89ad-1dcdb041609f" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c14beae7-d7f4-4a78-9bee-61a3ec1ee416.assettoscacsar" CURR_ARTIFACT_UUID = "c14beae7-d7f4-4a78-9bee-61a3ec1ee416.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-ExtZteVdu-csar.csar,0,c14beae7-d7f4-4a78-9bee-61a3ec1ee416.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=6235 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=0674e08d-83e3-49b4-8888-8df2cea4beb1 TargetServiceName=null EntryTimestamp=2021-08-11T16:15:16.216Z InvocationID=b6e4f30a-fd02-4751-afa6-57e5ce8da7ea ErrorCode=500 LogTimestamp=2021-08-11T16:15:22.451074Z ResponseDesc=Created Exiting. 2021-08-11T16:15:22.451Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:15:22.480Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=1e788507-ccb8-40ab-87ed-3c6653bcde4e InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:15:22.479801Z InvocationID=a0680128-39ae-4fb5-9d66-7f324ad79e04 LogTimestamp=2021-08-11T16:15:22.479998Z Entering 2021-08-11T16:15:22.480Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=1e788507-ccb8-40ab-87ed-3c6653bcde4e InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:15:22.479801Z InvocationID=a0680128-39ae-4fb5-9d66-7f324ad79e04 LogTimestamp=2021-08-11T16:15:22.479998Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:15:22.966Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f20aa66-e0ee-4445-a039-18b84b1f2dcf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:15:22.966Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f20aa66-e0ee-4445-a039-18b84b1f2dcf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6a75ee47 2021-08-11T16:15:22.967Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f20aa66-e0ee-4445-a039-18b84b1f2dcf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:15:22.967Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f20aa66-e0ee-4445-a039-18b84b1f2dcf PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:15:23.595Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:22.480Z InvocationID=d7255868-c83f-4fd0-ad83-ac164823e8d0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:15:23.595Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:22.480Z InvocationID=d7255868-c83f-4fd0-ad83-ac164823e8d0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:15:23.595Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:22.480Z InvocationID=d7255868-c83f-4fd0-ad83-ac164823e8d0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:15:23.595Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:22.480Z InvocationID=d7255868-c83f-4fd0-ad83-ac164823e8d0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:15:23.653Z [qtp2045766957-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:22.480Z InvocationID=d7255868-c83f-4fd0-ad83-ac164823e8d0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name ExtZteVl 2021-08-11T16:15:25.163Z [qtp2045766957-15] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ext ZTE VL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "79ef4826-8354-4936-91ab-a7f91e7b3a6a" INVARIANT_UUID = "79dff3f5-8020-4d31-a069-c5cebf38a9f7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.ext.zte.VL" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=79ef4826-8354-4936-91ab-a7f91e7b3a6a EntryTimestamp=2021-08-11T16:15:22.480Z InvocationID=d7255868-c83f-4fd0-ad83-ac164823e8d0 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:15:25.164Z [qtp2045766957-15] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ext ZTE VL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "79ef4826-8354-4936-91ab-a7f91e7b3a6a" INVARIANT_UUID = "79dff3f5-8020-4d31-a069-c5cebf38a9f7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.ext.zte.VL" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=79ef4826-8354-4936-91ab-a7f91e7b3a6a TargetServiceName=null EntryTimestamp=2021-08-11T16:15:22.480Z InvocationID=d7255868-c83f-4fd0-ad83-ac164823e8d0 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:15:25.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=afc4aba7-1056-4347-9021-3e0938b49b76 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:15:25.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=afc4aba7-1056-4347-9021-3e0938b49b76 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6a20d05a 2021-08-11T16:15:25.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=afc4aba7-1056-4347-9021-3e0938b49b76 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:15:25.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=afc4aba7-1056-4347-9021-3e0938b49b76 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:15:25.970Z [qtp2045766957-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "ext ZTE VL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "79ef4826-8354-4936-91ab-a7f91e7b3a6a" INVARIANT_UUID = "79dff3f5-8020-4d31-a069-c5cebf38a9f7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.ext.zte.VL" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=79ef4826-8354-4936-91ab-a7f91e7b3a6a TargetServiceName=null EntryTimestamp=2021-08-11T16:15:22.480Z InvocationID=d7255868-c83f-4fd0-ad83-ac164823e8d0 ErrorCode=500 lock resource with id 35331224-5f5e-4a22-bb35-6e5340d36e18 2021-08-11T16:15:27.781Z [qtp2045766957-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ext ZTE VL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "79ef4826-8354-4936-91ab-a7f91e7b3a6a" INVARIANT_UUID = "79dff3f5-8020-4d31-a069-c5cebf38a9f7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "35331224-5f5e-4a22-bb35-6e5340d36e18.assettoscatemplate" CURR_ARTIFACT_UUID = "35331224-5f5e-4a22-bb35-6e5340d36e18.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-ExtZteVl-template.yml,0,35331224-5f5e-4a22-bb35-6e5340d36e18.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a84caa22-bcb1-40d8-a39d-090d5c1b7ff3 TargetServiceName=null EntryTimestamp=2021-08-11T16:15:22.480Z InvocationID=d7255868-c83f-4fd0-ad83-ac164823e8d0 ErrorCode=500 Starting to create Csar for component ext ZTE VL 2021-08-11T16:15:27.987Z [qtp2045766957-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ext ZTE VL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "79ef4826-8354-4936-91ab-a7f91e7b3a6a" INVARIANT_UUID = "79dff3f5-8020-4d31-a069-c5cebf38a9f7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "35331224-5f5e-4a22-bb35-6e5340d36e18.assettoscatemplate" CURR_ARTIFACT_UUID = "35331224-5f5e-4a22-bb35-6e5340d36e18.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-ExtZteVl-template.yml,0,35331224-5f5e-4a22-bb35-6e5340d36e18.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a84caa22-bcb1-40d8-a39d-090d5c1b7ff3 TargetServiceName=null EntryTimestamp=2021-08-11T16:15:22.480Z InvocationID=d7255868-c83f-4fd0-ad83-ac164823e8d0 ErrorCode=500 Ended create Csar for component ext ZTE VL 2021-08-11T16:15:27.987Z [qtp2045766957-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ext ZTE VL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "79ef4826-8354-4936-91ab-a7f91e7b3a6a" INVARIANT_UUID = "79dff3f5-8020-4d31-a069-c5cebf38a9f7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "35331224-5f5e-4a22-bb35-6e5340d36e18.assettoscatemplate" CURR_ARTIFACT_UUID = "35331224-5f5e-4a22-bb35-6e5340d36e18.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-ExtZteVl-template.yml,0,35331224-5f5e-4a22-bb35-6e5340d36e18.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a84caa22-bcb1-40d8-a39d-090d5c1b7ff3 TargetServiceName=null EntryTimestamp=2021-08-11T16:15:22.480Z InvocationID=d7255868-c83f-4fd0-ad83-ac164823e8d0 ErrorCode=500 lock resource with id 35331224-5f5e-4a22-bb35-6e5340d36e18 2021-08-11T16:15:28.950Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=295484a7-9809-4859-b838-4b508d9a2984 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:15:28.950Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=295484a7-9809-4859-b838-4b508d9a2984 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@71d64c3c 2021-08-11T16:15:28.950Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=295484a7-9809-4859-b838-4b508d9a2984 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:15:28.950Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=295484a7-9809-4859-b838-4b508d9a2984 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:15:29.919Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "ext ZTE VL" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "79ef4826-8354-4936-91ab-a7f91e7b3a6a" INVARIANT_UUID = "79dff3f5-8020-4d31-a069-c5cebf38a9f7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "35331224-5f5e-4a22-bb35-6e5340d36e18.assettoscacsar" CURR_ARTIFACT_UUID = "35331224-5f5e-4a22-bb35-6e5340d36e18.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-ExtZteVl-csar.csar,0,35331224-5f5e-4a22-bb35-6e5340d36e18.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=7439 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=31ab450e-4312-4f23-aec1-b81de270b32b TargetServiceName=null EntryTimestamp=2021-08-11T16:15:22.480Z InvocationID=d7255868-c83f-4fd0-ad83-ac164823e8d0 ErrorCode=500 LogTimestamp=2021-08-11T16:15:29.919532Z ResponseDesc=Created Exiting. 2021-08-11T16:15:29.920Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:15:29.938Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=a7c5ea21-7788-4b33-ad95-439e643c2082 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:15:29.938122Z InvocationID=eb3e2b02-ae33-45c9-a473-699ec5e362be LogTimestamp=2021-08-11T16:15:29.938176Z Entering 2021-08-11T16:15:29.938Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=a7c5ea21-7788-4b33-ad95-439e643c2082 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:15:29.938122Z InvocationID=eb3e2b02-ae33-45c9-a473-699ec5e362be LogTimestamp=2021-08-11T16:15:29.938176Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:15:30.270Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:29.939Z InvocationID=c9a335d5-e7c9-48e0-ac01-3f6ba767979a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:15:30.271Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:29.939Z InvocationID=c9a335d5-e7c9-48e0-ac01-3f6ba767979a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:15:30.271Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:29.939Z InvocationID=c9a335d5-e7c9-48e0-ac01-3f6ba767979a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:15:30.271Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:29.939Z InvocationID=c9a335d5-e7c9-48e0-ac01-3f6ba767979a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:15:30.285Z [qtp2045766957-18] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=Create Resource - validateLifecycleTypesCreate ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2021-08-11T16:15:29.939Z InvocationID=c9a335d5-e7c9-48e0-ac01-3f6ba767979a ErrorCode=400 Interface tosca.interfaces.nfv.nslcm required is missing 2021-08-11T16:15:30.285Z [qtp2045766957-18] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=Create Resource - validateLifecycleTypesCreate ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2021-08-11T16:15:29.939Z InvocationID=c9a335d5-e7c9-48e0-ac01-3f6ba767979a ErrorCode=500 Operation towards database failed. 2021-08-11T16:15:30.285Z [qtp2045766957-18] WARN o.o.s.e.AbstractSdncException RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2021-08-11T16:15:29.939Z InvocationID=c9a335d5-e7c9-48e0-ac01-3f6ba767979a ErrorCode=900 2021-08-11T16:15:30.288Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "" INVARIANT_UUID = "" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "400" DESC = "SVC4134: Error: Invalid Content. Missing interface life-cycle type." TOSCA_NODE_TYPE = "tosca.nodes.nfv.NS" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2021-08-11T16:15:29.939Z InvocationID=c9a335d5-e7c9-48e0-ac01-3f6ba767979a ErrorCode=900 lock resource with name Ns 2021-08-11T16:15:31.862Z [qtp2045766957-18] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "a6a35627-294a-4f3c-a892-e2f5bb303ca3" INVARIANT_UUID = "cc9a00a8-135c-4bbb-a261-ed8f9c0bba70" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NS" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity= ErrorCategory=WARN ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a6a35627-294a-4f3c-a892-e2f5bb303ca3 TargetServiceName=null EntryTimestamp=2021-08-11T16:15:29.939Z InvocationID=c9a335d5-e7c9-48e0-ac01-3f6ba767979a ErrorCode=900 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:15:31.863Z [qtp2045766957-18] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "a6a35627-294a-4f3c-a892-e2f5bb303ca3" INVARIANT_UUID = "cc9a00a8-135c-4bbb-a261-ed8f9c0bba70" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NS" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a6a35627-294a-4f3c-a892-e2f5bb303ca3 TargetServiceName=null EntryTimestamp=2021-08-11T16:15:29.939Z InvocationID=c9a335d5-e7c9-48e0-ac01-3f6ba767979a ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:15:31.947Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aacea0e0-a5c3-46ed-a109-00c283a77a4d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:15:31.947Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aacea0e0-a5c3-46ed-a109-00c283a77a4d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@56d7d99 2021-08-11T16:15:31.947Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aacea0e0-a5c3-46ed-a109-00c283a77a4d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:15:31.947Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aacea0e0-a5c3-46ed-a109-00c283a77a4d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:15:32.393Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "a6a35627-294a-4f3c-a892-e2f5bb303ca3" INVARIANT_UUID = "cc9a00a8-135c-4bbb-a261-ed8f9c0bba70" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NS" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a6a35627-294a-4f3c-a892-e2f5bb303ca3 TargetServiceName=null EntryTimestamp=2021-08-11T16:15:29.939Z InvocationID=c9a335d5-e7c9-48e0-ac01-3f6ba767979a ErrorCode=500 lock resource with id c92410eb-97df-4378-9616-a7005a777f41 2021-08-11T16:15:33.305Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:15:33.311Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:15:34.011Z [qtp2045766957-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "a6a35627-294a-4f3c-a892-e2f5bb303ca3" INVARIANT_UUID = "cc9a00a8-135c-4bbb-a261-ed8f9c0bba70" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c92410eb-97df-4378-9616-a7005a777f41.assettoscatemplate" CURR_ARTIFACT_UUID = "c92410eb-97df-4378-9616-a7005a777f41.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Ns-template.yml,0,c92410eb-97df-4378-9616-a7005a777f41.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d0a3cf09-fa21-41b7-b815-0a29685b06ac TargetServiceName=null EntryTimestamp=2021-08-11T16:15:29.939Z InvocationID=c9a335d5-e7c9-48e0-ac01-3f6ba767979a ErrorCode=500 Starting to create Csar for component NS 2021-08-11T16:15:34.254Z [qtp2045766957-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "a6a35627-294a-4f3c-a892-e2f5bb303ca3" INVARIANT_UUID = "cc9a00a8-135c-4bbb-a261-ed8f9c0bba70" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c92410eb-97df-4378-9616-a7005a777f41.assettoscatemplate" CURR_ARTIFACT_UUID = "c92410eb-97df-4378-9616-a7005a777f41.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Ns-template.yml,0,c92410eb-97df-4378-9616-a7005a777f41.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d0a3cf09-fa21-41b7-b815-0a29685b06ac TargetServiceName=null EntryTimestamp=2021-08-11T16:15:29.939Z InvocationID=c9a335d5-e7c9-48e0-ac01-3f6ba767979a ErrorCode=500 Ended create Csar for component NS 2021-08-11T16:15:34.254Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "a6a35627-294a-4f3c-a892-e2f5bb303ca3" INVARIANT_UUID = "cc9a00a8-135c-4bbb-a261-ed8f9c0bba70" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c92410eb-97df-4378-9616-a7005a777f41.assettoscatemplate" CURR_ARTIFACT_UUID = "c92410eb-97df-4378-9616-a7005a777f41.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Ns-template.yml,0,c92410eb-97df-4378-9616-a7005a777f41.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d0a3cf09-fa21-41b7-b815-0a29685b06ac TargetServiceName=null EntryTimestamp=2021-08-11T16:15:29.939Z InvocationID=c9a335d5-e7c9-48e0-ac01-3f6ba767979a ErrorCode=500 lock resource with id c92410eb-97df-4378-9616-a7005a777f41 2021-08-11T16:15:34.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb55d1d6-a920-4b08-9181-ffb2aab93c0e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:15:34.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb55d1d6-a920-4b08-9181-ffb2aab93c0e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@40afbca5 2021-08-11T16:15:34.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb55d1d6-a920-4b08-9181-ffb2aab93c0e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:15:34.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb55d1d6-a920-4b08-9181-ffb2aab93c0e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:15:35.941Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "a6a35627-294a-4f3c-a892-e2f5bb303ca3" INVARIANT_UUID = "cc9a00a8-135c-4bbb-a261-ed8f9c0bba70" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c92410eb-97df-4378-9616-a7005a777f41.assettoscacsar" CURR_ARTIFACT_UUID = "c92410eb-97df-4378-9616-a7005a777f41.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Ns-csar.csar,0,c92410eb-97df-4378-9616-a7005a777f41.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=6001 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d4a8ee15-e845-4fab-8384-0611be96aad9 TargetServiceName=null EntryTimestamp=2021-08-11T16:15:29.939Z InvocationID=c9a335d5-e7c9-48e0-ac01-3f6ba767979a ErrorCode=500 LogTimestamp=2021-08-11T16:15:35.940996Z ResponseDesc=Created Exiting. 2021-08-11T16:15:35.941Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:15:35.957Z [qtp2045766957-11] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=ceb3b5c2-6c55-4bd7-bc5d-7ea9c353e130 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:15:35.957462Z InvocationID=fe5a7ddc-76b7-4cf9-bdeb-3047ac5cd08a LogTimestamp=2021-08-11T16:15:35.957496Z Entering 2021-08-11T16:15:35.957Z [qtp2045766957-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=ceb3b5c2-6c55-4bd7-bc5d-7ea9c353e130 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:15:35.957462Z InvocationID=fe5a7ddc-76b7-4cf9-bdeb-3047ac5cd08a LogTimestamp=2021-08-11T16:15:35.957496Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:15:36.312Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:35.958Z InvocationID=20d6d433-1320-49cf-954a-8fbcee5a9755 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:15:36.312Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:35.958Z InvocationID=20d6d433-1320-49cf-954a-8fbcee5a9755 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:15:36.312Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:35.958Z InvocationID=20d6d433-1320-49cf-954a-8fbcee5a9755 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:15:36.312Z [qtp2045766957-11] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:35.958Z InvocationID=20d6d433-1320-49cf-954a-8fbcee5a9755 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:15:36.339Z [qtp2045766957-11] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:35.958Z InvocationID=20d6d433-1320-49cf-954a-8fbcee5a9755 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Nsd 2021-08-11T16:15:37.759Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02290fce-89bd-4445-b953-728029f6dc68 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:15:37.759Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02290fce-89bd-4445-b953-728029f6dc68 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2ce116ea 2021-08-11T16:15:37.759Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02290fce-89bd-4445-b953-728029f6dc68 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:15:37.759Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02290fce-89bd-4445-b953-728029f6dc68 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:15:37.972Z [qtp2045766957-11] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NSD" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "77860363-90fe-4167-97e3-fb7f31d8278b" INVARIANT_UUID = "eb72944e-317b-4136-a71c-4e0984507d52" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.NSD" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=77860363-90fe-4167-97e3-fb7f31d8278b EntryTimestamp=2021-08-11T16:15:35.958Z InvocationID=20d6d433-1320-49cf-954a-8fbcee5a9755 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:15:37.973Z [qtp2045766957-11] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NSD" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "77860363-90fe-4167-97e3-fb7f31d8278b" INVARIANT_UUID = "eb72944e-317b-4136-a71c-4e0984507d52" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.NSD" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=77860363-90fe-4167-97e3-fb7f31d8278b TargetServiceName=null EntryTimestamp=2021-08-11T16:15:35.958Z InvocationID=20d6d433-1320-49cf-954a-8fbcee5a9755 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:15:38.858Z [qtp2045766957-11] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NSD" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "77860363-90fe-4167-97e3-fb7f31d8278b" INVARIANT_UUID = "eb72944e-317b-4136-a71c-4e0984507d52" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.NSD" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=77860363-90fe-4167-97e3-fb7f31d8278b TargetServiceName=null EntryTimestamp=2021-08-11T16:15:35.958Z InvocationID=20d6d433-1320-49cf-954a-8fbcee5a9755 ErrorCode=500 lock resource with id 7102cfd5-c008-4fae-a31a-8d3e88fe8f18 2021-08-11T16:15:40.855Z [qtp2045766957-11] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NSD" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "77860363-90fe-4167-97e3-fb7f31d8278b" INVARIANT_UUID = "eb72944e-317b-4136-a71c-4e0984507d52" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "7102cfd5-c008-4fae-a31a-8d3e88fe8f18.assettoscatemplate" CURR_ARTIFACT_UUID = "7102cfd5-c008-4fae-a31a-8d3e88fe8f18.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Nsd-template.yml,0,7102cfd5-c008-4fae-a31a-8d3e88fe8f18.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=0d59b348-33b8-4856-bc17-b15f1f222e87 TargetServiceName=null EntryTimestamp=2021-08-11T16:15:35.958Z InvocationID=20d6d433-1320-49cf-954a-8fbcee5a9755 ErrorCode=500 Starting to create Csar for component NSD 2021-08-11T16:15:40.951Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=490529bd-48bd-48f6-af0b-8bfbf37303bb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:15:40.951Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=490529bd-48bd-48f6-af0b-8bfbf37303bb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6f1518e6 2021-08-11T16:15:40.951Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=490529bd-48bd-48f6-af0b-8bfbf37303bb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:15:40.951Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=490529bd-48bd-48f6-af0b-8bfbf37303bb PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:15:41.045Z [qtp2045766957-11] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NSD" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "77860363-90fe-4167-97e3-fb7f31d8278b" INVARIANT_UUID = "eb72944e-317b-4136-a71c-4e0984507d52" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "7102cfd5-c008-4fae-a31a-8d3e88fe8f18.assettoscatemplate" CURR_ARTIFACT_UUID = "7102cfd5-c008-4fae-a31a-8d3e88fe8f18.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Nsd-template.yml,0,7102cfd5-c008-4fae-a31a-8d3e88fe8f18.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=0d59b348-33b8-4856-bc17-b15f1f222e87 TargetServiceName=null EntryTimestamp=2021-08-11T16:15:35.958Z InvocationID=20d6d433-1320-49cf-954a-8fbcee5a9755 ErrorCode=500 Ended create Csar for component NSD 2021-08-11T16:15:41.046Z [qtp2045766957-11] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NSD" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "77860363-90fe-4167-97e3-fb7f31d8278b" INVARIANT_UUID = "eb72944e-317b-4136-a71c-4e0984507d52" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "7102cfd5-c008-4fae-a31a-8d3e88fe8f18.assettoscatemplate" CURR_ARTIFACT_UUID = "7102cfd5-c008-4fae-a31a-8d3e88fe8f18.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Nsd-template.yml,0,7102cfd5-c008-4fae-a31a-8d3e88fe8f18.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=0d59b348-33b8-4856-bc17-b15f1f222e87 TargetServiceName=null EntryTimestamp=2021-08-11T16:15:35.958Z InvocationID=20d6d433-1320-49cf-954a-8fbcee5a9755 ErrorCode=500 lock resource with id 7102cfd5-c008-4fae-a31a-8d3e88fe8f18 2021-08-11T16:15:42.697Z [qtp2045766957-11] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NSD" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "77860363-90fe-4167-97e3-fb7f31d8278b" INVARIANT_UUID = "eb72944e-317b-4136-a71c-4e0984507d52" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "7102cfd5-c008-4fae-a31a-8d3e88fe8f18.assettoscacsar" CURR_ARTIFACT_UUID = "7102cfd5-c008-4fae-a31a-8d3e88fe8f18.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Nsd-csar.csar,0,7102cfd5-c008-4fae-a31a-8d3e88fe8f18.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=6739 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=13457e89-f50b-4903-966b-8fadd587243a TargetServiceName=null EntryTimestamp=2021-08-11T16:15:35.958Z InvocationID=20d6d433-1320-49cf-954a-8fbcee5a9755 ErrorCode=500 LogTimestamp=2021-08-11T16:15:42.697005Z ResponseDesc=Created Exiting. 2021-08-11T16:15:42.698Z [qtp2045766957-11] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:15:42.717Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=452a5637-4c3b-41fe-acaa-0bc3ae2dabc5 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:15:42.716967Z InvocationID=810268c7-4c75-466f-8f63-58886700815d LogTimestamp=2021-08-11T16:15:42.717030Z Entering 2021-08-11T16:15:42.717Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=452a5637-4c3b-41fe-acaa-0bc3ae2dabc5 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:15:42.716967Z InvocationID=810268c7-4c75-466f-8f63-58886700815d LogTimestamp=2021-08-11T16:15:42.717030Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:15:43.168Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:42.717Z InvocationID=086b29f0-ae8b-4fc4-be90-6664099ebdfc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:15:43.168Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:42.717Z InvocationID=086b29f0-ae8b-4fc4-be90-6664099ebdfc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:15:43.168Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:42.717Z InvocationID=086b29f0-ae8b-4fc4-be90-6664099ebdfc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:15:43.168Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:42.717Z InvocationID=086b29f0-ae8b-4fc4-be90-6664099ebdfc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:15:43.188Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:42.717Z InvocationID=086b29f0-ae8b-4fc4-be90-6664099ebdfc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Nsvirtuallink 2021-08-11T16:15:43.700Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01ef4735-afe0-4d8c-8917-5d1a19a4aab1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:15:43.700Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01ef4735-afe0-4d8c-8917-5d1a19a4aab1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@17a3128 2021-08-11T16:15:43.700Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01ef4735-afe0-4d8c-8917-5d1a19a4aab1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:15:43.700Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01ef4735-afe0-4d8c-8917-5d1a19a4aab1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:15:44.856Z [qtp2045766957-18] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NsVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "47a7e743-961d-4b80-8b44-fafece5a8858" INVARIANT_UUID = "4b7ad971-f9e2-4442-80a2-a6af38b31bc0" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NsVirtualLink" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=47a7e743-961d-4b80-8b44-fafece5a8858 EntryTimestamp=2021-08-11T16:15:42.717Z InvocationID=086b29f0-ae8b-4fc4-be90-6664099ebdfc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:15:44.857Z [qtp2045766957-18] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NsVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "47a7e743-961d-4b80-8b44-fafece5a8858" INVARIANT_UUID = "4b7ad971-f9e2-4442-80a2-a6af38b31bc0" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NsVirtualLink" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=47a7e743-961d-4b80-8b44-fafece5a8858 TargetServiceName=null EntryTimestamp=2021-08-11T16:15:42.717Z InvocationID=086b29f0-ae8b-4fc4-be90-6664099ebdfc ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:15:45.121Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NsVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "47a7e743-961d-4b80-8b44-fafece5a8858" INVARIANT_UUID = "4b7ad971-f9e2-4442-80a2-a6af38b31bc0" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NsVirtualLink" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=47a7e743-961d-4b80-8b44-fafece5a8858 TargetServiceName=null EntryTimestamp=2021-08-11T16:15:42.717Z InvocationID=086b29f0-ae8b-4fc4-be90-6664099ebdfc ErrorCode=500 lock resource with id 70413ffc-1bc9-43e6-b8ed-a8faca8b6dcf 2021-08-11T16:15:46.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a007a32c-5044-4ddb-9deb-267134f03ebd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:15:46.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a007a32c-5044-4ddb-9deb-267134f03ebd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3c0def36 2021-08-11T16:15:46.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a007a32c-5044-4ddb-9deb-267134f03ebd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:15:46.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a007a32c-5044-4ddb-9deb-267134f03ebd PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:15:46.783Z [qtp2045766957-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NsVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "47a7e743-961d-4b80-8b44-fafece5a8858" INVARIANT_UUID = "4b7ad971-f9e2-4442-80a2-a6af38b31bc0" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "70413ffc-1bc9-43e6-b8ed-a8faca8b6dcf.assettoscatemplate" CURR_ARTIFACT_UUID = "70413ffc-1bc9-43e6-b8ed-a8faca8b6dcf.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Nsvirtuallink-template.yml,0,70413ffc-1bc9-43e6-b8ed-a8faca8b6dcf.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ee3c1660-5ad7-4330-bbcc-6327d879b502 TargetServiceName=null EntryTimestamp=2021-08-11T16:15:42.717Z InvocationID=086b29f0-ae8b-4fc4-be90-6664099ebdfc ErrorCode=500 Starting to create Csar for component NsVirtualLink 2021-08-11T16:15:46.933Z [qtp2045766957-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NsVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "47a7e743-961d-4b80-8b44-fafece5a8858" INVARIANT_UUID = "4b7ad971-f9e2-4442-80a2-a6af38b31bc0" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "70413ffc-1bc9-43e6-b8ed-a8faca8b6dcf.assettoscatemplate" CURR_ARTIFACT_UUID = "70413ffc-1bc9-43e6-b8ed-a8faca8b6dcf.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Nsvirtuallink-template.yml,0,70413ffc-1bc9-43e6-b8ed-a8faca8b6dcf.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ee3c1660-5ad7-4330-bbcc-6327d879b502 TargetServiceName=null EntryTimestamp=2021-08-11T16:15:42.717Z InvocationID=086b29f0-ae8b-4fc4-be90-6664099ebdfc ErrorCode=500 Ended create Csar for component NsVirtualLink 2021-08-11T16:15:46.933Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NsVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "47a7e743-961d-4b80-8b44-fafece5a8858" INVARIANT_UUID = "4b7ad971-f9e2-4442-80a2-a6af38b31bc0" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "70413ffc-1bc9-43e6-b8ed-a8faca8b6dcf.assettoscatemplate" CURR_ARTIFACT_UUID = "70413ffc-1bc9-43e6-b8ed-a8faca8b6dcf.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Nsvirtuallink-template.yml,0,70413ffc-1bc9-43e6-b8ed-a8faca8b6dcf.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ee3c1660-5ad7-4330-bbcc-6327d879b502 TargetServiceName=null EntryTimestamp=2021-08-11T16:15:42.717Z InvocationID=086b29f0-ae8b-4fc4-be90-6664099ebdfc ErrorCode=500 lock resource with id 70413ffc-1bc9-43e6-b8ed-a8faca8b6dcf 2021-08-11T16:15:48.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:15:48.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:15:48.859Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NsVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "47a7e743-961d-4b80-8b44-fafece5a8858" INVARIANT_UUID = "4b7ad971-f9e2-4442-80a2-a6af38b31bc0" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "70413ffc-1bc9-43e6-b8ed-a8faca8b6dcf.assettoscacsar" CURR_ARTIFACT_UUID = "70413ffc-1bc9-43e6-b8ed-a8faca8b6dcf.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Nsvirtuallink-csar.csar,0,70413ffc-1bc9-43e6-b8ed-a8faca8b6dcf.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=6142 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=5e24862e-f620-46c1-b78f-3cf71f1460ac TargetServiceName=null EntryTimestamp=2021-08-11T16:15:42.717Z InvocationID=086b29f0-ae8b-4fc4-be90-6664099ebdfc ErrorCode=500 LogTimestamp=2021-08-11T16:15:48.859283Z ResponseDesc=Created Exiting. 2021-08-11T16:15:48.860Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:15:48.877Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=5e6fe61c-d91a-4c2d-9452-05683ce8b09b InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:15:48.876977Z InvocationID=f5526889-a588-4dfb-b430-d573680b1590 LogTimestamp=2021-08-11T16:15:48.877035Z Entering 2021-08-11T16:15:48.877Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=5e6fe61c-d91a-4c2d-9452-05683ce8b09b InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:15:48.876977Z InvocationID=f5526889-a588-4dfb-b430-d573680b1590 LogTimestamp=2021-08-11T16:15:48.877035Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:15:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:15:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:15:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:15:50.154Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:48.877Z InvocationID=81c4124e-d5da-49a2-8ac5-ceb8ebb0f1d8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:15:50.154Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:48.877Z InvocationID=81c4124e-d5da-49a2-8ac5-ceb8ebb0f1d8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:15:50.154Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:48.877Z InvocationID=81c4124e-d5da-49a2-8ac5-ceb8ebb0f1d8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:15:50.154Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:48.877Z InvocationID=81c4124e-d5da-49a2-8ac5-ceb8ebb0f1d8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:15:50.163Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f629f7c7-0ba2-4a7a-99b6-a86948818b29 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:15:50.163Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f629f7c7-0ba2-4a7a-99b6-a86948818b29 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@18fff81b 2021-08-11T16:15:50.163Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f629f7c7-0ba2-4a7a-99b6-a86948818b29 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:15:50.163Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f629f7c7-0ba2-4a7a-99b6-a86948818b29 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:15:50.458Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:48.877Z InvocationID=81c4124e-d5da-49a2-8ac5-ceb8ebb0f1d8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Vdu 2021-08-11T16:15:53.478Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14a31954-cb24-4d28-8bdf-d3f470aebd5a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:15:53.478Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14a31954-cb24-4d28-8bdf-d3f470aebd5a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5f6a11ca 2021-08-11T16:15:53.478Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14a31954-cb24-4d28-8bdf-d3f470aebd5a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:15:53.478Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14a31954-cb24-4d28-8bdf-d3f470aebd5a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:15:53.655Z [qtp2045766957-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "06d0449a-93c7-4ef3-b640-0431d4d95a59" INVARIANT_UUID = "261eea01-55ba-433e-86a8-42799c034f38" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VDU" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=06d0449a-93c7-4ef3-b640-0431d4d95a59 EntryTimestamp=2021-08-11T16:15:48.877Z InvocationID=81c4124e-d5da-49a2-8ac5-ceb8ebb0f1d8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:15:53.655Z [qtp2045766957-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "06d0449a-93c7-4ef3-b640-0431d4d95a59" INVARIANT_UUID = "261eea01-55ba-433e-86a8-42799c034f38" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VDU" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=06d0449a-93c7-4ef3-b640-0431d4d95a59 TargetServiceName=null EntryTimestamp=2021-08-11T16:15:48.877Z InvocationID=81c4124e-d5da-49a2-8ac5-ceb8ebb0f1d8 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:15:55.150Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "06d0449a-93c7-4ef3-b640-0431d4d95a59" INVARIANT_UUID = "261eea01-55ba-433e-86a8-42799c034f38" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VDU" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=06d0449a-93c7-4ef3-b640-0431d4d95a59 TargetServiceName=null EntryTimestamp=2021-08-11T16:15:48.877Z InvocationID=81c4124e-d5da-49a2-8ac5-ceb8ebb0f1d8 ErrorCode=500 lock resource with id 57981c62-9b4a-4dfd-bc8c-8ba9a76764b8 2021-08-11T16:15:56.553Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8e0ba0d9-1df7-49ae-8a97-08038208fa53 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:15:56.553Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8e0ba0d9-1df7-49ae-8a97-08038208fa53 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@f49887e 2021-08-11T16:15:56.553Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8e0ba0d9-1df7-49ae-8a97-08038208fa53 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:15:56.553Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8e0ba0d9-1df7-49ae-8a97-08038208fa53 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:15:57.672Z [qtp2045766957-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "06d0449a-93c7-4ef3-b640-0431d4d95a59" INVARIANT_UUID = "261eea01-55ba-433e-86a8-42799c034f38" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "57981c62-9b4a-4dfd-bc8c-8ba9a76764b8.assettoscatemplate" CURR_ARTIFACT_UUID = "57981c62-9b4a-4dfd-bc8c-8ba9a76764b8.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vdu-template.yml,0,57981c62-9b4a-4dfd-bc8c-8ba9a76764b8.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=755ceca1-cb51-45ba-8220-e1a12efac6b4 TargetServiceName=null EntryTimestamp=2021-08-11T16:15:48.877Z InvocationID=81c4124e-d5da-49a2-8ac5-ceb8ebb0f1d8 ErrorCode=500 Starting to create Csar for component VDU 2021-08-11T16:15:57.910Z [qtp2045766957-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "06d0449a-93c7-4ef3-b640-0431d4d95a59" INVARIANT_UUID = "261eea01-55ba-433e-86a8-42799c034f38" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "57981c62-9b4a-4dfd-bc8c-8ba9a76764b8.assettoscatemplate" CURR_ARTIFACT_UUID = "57981c62-9b4a-4dfd-bc8c-8ba9a76764b8.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vdu-template.yml,0,57981c62-9b4a-4dfd-bc8c-8ba9a76764b8.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=755ceca1-cb51-45ba-8220-e1a12efac6b4 TargetServiceName=null EntryTimestamp=2021-08-11T16:15:48.877Z InvocationID=81c4124e-d5da-49a2-8ac5-ceb8ebb0f1d8 ErrorCode=500 Ended create Csar for component VDU 2021-08-11T16:15:57.910Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "06d0449a-93c7-4ef3-b640-0431d4d95a59" INVARIANT_UUID = "261eea01-55ba-433e-86a8-42799c034f38" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "57981c62-9b4a-4dfd-bc8c-8ba9a76764b8.assettoscatemplate" CURR_ARTIFACT_UUID = "57981c62-9b4a-4dfd-bc8c-8ba9a76764b8.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vdu-template.yml,0,57981c62-9b4a-4dfd-bc8c-8ba9a76764b8.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=755ceca1-cb51-45ba-8220-e1a12efac6b4 TargetServiceName=null EntryTimestamp=2021-08-11T16:15:48.877Z InvocationID=81c4124e-d5da-49a2-8ac5-ceb8ebb0f1d8 ErrorCode=500 lock resource with id 57981c62-9b4a-4dfd-bc8c-8ba9a76764b8 2021-08-11T16:15:59.371Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=174d95ef-5ff6-4515-a989-d2c3c83c7698 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:15:59.371Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=174d95ef-5ff6-4515-a989-d2c3c83c7698 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5c6ea2fe 2021-08-11T16:15:59.371Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=174d95ef-5ff6-4515-a989-d2c3c83c7698 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:15:59.371Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=174d95ef-5ff6-4515-a989-d2c3c83c7698 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:15:59.882Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "06d0449a-93c7-4ef3-b640-0431d4d95a59" INVARIANT_UUID = "261eea01-55ba-433e-86a8-42799c034f38" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "57981c62-9b4a-4dfd-bc8c-8ba9a76764b8.assettoscacsar" CURR_ARTIFACT_UUID = "57981c62-9b4a-4dfd-bc8c-8ba9a76764b8.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Vdu-csar.csar,0,57981c62-9b4a-4dfd-bc8c-8ba9a76764b8.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=11005 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=cf05c9f6-cc28-47ea-8520-4a806288a8aa TargetServiceName=null EntryTimestamp=2021-08-11T16:15:48.877Z InvocationID=81c4124e-d5da-49a2-8ac5-ceb8ebb0f1d8 ErrorCode=500 LogTimestamp=2021-08-11T16:15:59.882234Z ResponseDesc=Created Exiting. 2021-08-11T16:15:59.883Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:15:59.971Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=fe6eaf72-5022-482c-a8fd-05177794c9a6 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:15:59.970872Z InvocationID=b589419b-f494-4858-9850-26ec10534af5 LogTimestamp=2021-08-11T16:15:59.970935Z Entering 2021-08-11T16:15:59.971Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=fe6eaf72-5022-482c-a8fd-05177794c9a6 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:15:59.970872Z InvocationID=b589419b-f494-4858-9850-26ec10534af5 LogTimestamp=2021-08-11T16:15:59.970935Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:16:01.875Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:59.971Z InvocationID=d1e87d64-5732-4bd0-8767-ff38397f4dc8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:01.875Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:59.971Z InvocationID=d1e87d64-5732-4bd0-8767-ff38397f4dc8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:01.875Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:59.971Z InvocationID=d1e87d64-5732-4bd0-8767-ff38397f4dc8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:01.875Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:59.971Z InvocationID=d1e87d64-5732-4bd0-8767-ff38397f4dc8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:02.173Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:15:59.971Z InvocationID=d1e87d64-5732-4bd0-8767-ff38397f4dc8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name VduCompute 2021-08-11T16:16:02.285Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd2b0bea-ad75-4091-8afd-595056914509 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:16:02.285Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd2b0bea-ad75-4091-8afd-595056914509 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3b36b648 2021-08-11T16:16:02.286Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd2b0bea-ad75-4091-8afd-595056914509 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:16:02.286Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd2b0bea-ad75-4091-8afd-595056914509 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:16:03.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:16:03.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:16:03.785Z [qtp2045766957-18] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "380fb32c-7e78-45d8-9833-851d7a844cf2" INVARIANT_UUID = "a9e1a9b1-f8e2-48e8-89d4-80b004df2b96" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.Compute" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=380fb32c-7e78-45d8-9833-851d7a844cf2 EntryTimestamp=2021-08-11T16:15:59.971Z InvocationID=d1e87d64-5732-4bd0-8767-ff38397f4dc8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:16:03.785Z [qtp2045766957-18] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "380fb32c-7e78-45d8-9833-851d7a844cf2" INVARIANT_UUID = "a9e1a9b1-f8e2-48e8-89d4-80b004df2b96" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.Compute" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=380fb32c-7e78-45d8-9833-851d7a844cf2 TargetServiceName=null EntryTimestamp=2021-08-11T16:15:59.971Z InvocationID=d1e87d64-5732-4bd0-8767-ff38397f4dc8 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:16:04.264Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:16:04.274Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "380fb32c-7e78-45d8-9833-851d7a844cf2" INVARIANT_UUID = "a9e1a9b1-f8e2-48e8-89d4-80b004df2b96" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.Compute" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=380fb32c-7e78-45d8-9833-851d7a844cf2 TargetServiceName=null EntryTimestamp=2021-08-11T16:15:59.971Z InvocationID=d1e87d64-5732-4bd0-8767-ff38397f4dc8 ErrorCode=500 lock resource with id 05b4e29f-606e-40e5-a4fd-bafd6aeb3b7a 2021-08-11T16:16:05.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=110597bf-a13d-40e5-84eb-5e252a73ce05 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:16:05.074Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=110597bf-a13d-40e5-84eb-5e252a73ce05 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@345d14a5 2021-08-11T16:16:05.074Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=110597bf-a13d-40e5-84eb-5e252a73ce05 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:16:05.074Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=110597bf-a13d-40e5-84eb-5e252a73ce05 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:16:06.006Z [qtp2045766957-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "380fb32c-7e78-45d8-9833-851d7a844cf2" INVARIANT_UUID = "a9e1a9b1-f8e2-48e8-89d4-80b004df2b96" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "05b4e29f-606e-40e5-a4fd-bafd6aeb3b7a.assettoscatemplate" CURR_ARTIFACT_UUID = "05b4e29f-606e-40e5-a4fd-bafd6aeb3b7a.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduCompute-template.yml,0,05b4e29f-606e-40e5-a4fd-bafd6aeb3b7a.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b508aff1-6062-4b17-bfca-df414af329c8 TargetServiceName=null EntryTimestamp=2021-08-11T16:15:59.971Z InvocationID=d1e87d64-5732-4bd0-8767-ff38397f4dc8 ErrorCode=500 Starting to create Csar for component VDU Compute 2021-08-11T16:16:06.144Z [qtp2045766957-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "380fb32c-7e78-45d8-9833-851d7a844cf2" INVARIANT_UUID = "a9e1a9b1-f8e2-48e8-89d4-80b004df2b96" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "05b4e29f-606e-40e5-a4fd-bafd6aeb3b7a.assettoscatemplate" CURR_ARTIFACT_UUID = "05b4e29f-606e-40e5-a4fd-bafd6aeb3b7a.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduCompute-template.yml,0,05b4e29f-606e-40e5-a4fd-bafd6aeb3b7a.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b508aff1-6062-4b17-bfca-df414af329c8 TargetServiceName=null EntryTimestamp=2021-08-11T16:15:59.971Z InvocationID=d1e87d64-5732-4bd0-8767-ff38397f4dc8 ErrorCode=500 Ended create Csar for component VDU Compute 2021-08-11T16:16:06.144Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "380fb32c-7e78-45d8-9833-851d7a844cf2" INVARIANT_UUID = "a9e1a9b1-f8e2-48e8-89d4-80b004df2b96" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "05b4e29f-606e-40e5-a4fd-bafd6aeb3b7a.assettoscatemplate" CURR_ARTIFACT_UUID = "05b4e29f-606e-40e5-a4fd-bafd6aeb3b7a.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduCompute-template.yml,0,05b4e29f-606e-40e5-a4fd-bafd6aeb3b7a.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b508aff1-6062-4b17-bfca-df414af329c8 TargetServiceName=null EntryTimestamp=2021-08-11T16:15:59.971Z InvocationID=d1e87d64-5732-4bd0-8767-ff38397f4dc8 ErrorCode=500 lock resource with id 05b4e29f-606e-40e5-a4fd-bafd6aeb3b7a 2021-08-11T16:16:07.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=053ddaca-a181-4530-a57c-1934e9225c1d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:16:07.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=053ddaca-a181-4530-a57c-1934e9225c1d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@77147028 2021-08-11T16:16:07.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=053ddaca-a181-4530-a57c-1934e9225c1d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:16:07.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=053ddaca-a181-4530-a57c-1934e9225c1d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:16:07.795Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "380fb32c-7e78-45d8-9833-851d7a844cf2" INVARIANT_UUID = "a9e1a9b1-f8e2-48e8-89d4-80b004df2b96" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "05b4e29f-606e-40e5-a4fd-bafd6aeb3b7a.assettoscacsar" CURR_ARTIFACT_UUID = "05b4e29f-606e-40e5-a4fd-bafd6aeb3b7a.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VduCompute-csar.csar,0,05b4e29f-606e-40e5-a4fd-bafd6aeb3b7a.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=7823 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=fe703701-2801-4fea-a2bc-ff31e5449664 TargetServiceName=null EntryTimestamp=2021-08-11T16:15:59.971Z InvocationID=d1e87d64-5732-4bd0-8767-ff38397f4dc8 ErrorCode=500 LogTimestamp=2021-08-11T16:16:07.794955Z ResponseDesc=Created Exiting. 2021-08-11T16:16:07.795Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:16:07.852Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=27cc1680-0c67-4ba6-8da1-96a653ff91da InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:16:07.852133Z InvocationID=c11f374a-f0ec-4530-95ac-330a08a0786f LogTimestamp=2021-08-11T16:16:07.852193Z Entering 2021-08-11T16:16:07.852Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=27cc1680-0c67-4ba6-8da1-96a653ff91da InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:16:07.852133Z InvocationID=c11f374a-f0ec-4530-95ac-330a08a0786f LogTimestamp=2021-08-11T16:16:07.852193Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:16:08.196Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:07.852Z InvocationID=791b5687-5ba6-4792-8468-3ecd25a2bd0c PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:08.196Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:07.852Z InvocationID=791b5687-5ba6-4792-8468-3ecd25a2bd0c PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:08.196Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:07.852Z InvocationID=791b5687-5ba6-4792-8468-3ecd25a2bd0c PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:08.196Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:07.852Z InvocationID=791b5687-5ba6-4792-8468-3ecd25a2bd0c PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:08.196Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:07.852Z InvocationID=791b5687-5ba6-4792-8468-3ecd25a2bd0c PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Cp 2021-08-11T16:16:09.493Z [qtp2045766957-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Cp" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "69e5f764-c037-48d7-87a5-929bfa84f0c6" INVARIANT_UUID = "4566db8d-ca11-4752-bcf6-522ab61931fb" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Cp" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=69e5f764-c037-48d7-87a5-929bfa84f0c6 EntryTimestamp=2021-08-11T16:16:07.852Z InvocationID=791b5687-5ba6-4792-8468-3ecd25a2bd0c PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:16:09.494Z [qtp2045766957-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Cp" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "69e5f764-c037-48d7-87a5-929bfa84f0c6" INVARIANT_UUID = "4566db8d-ca11-4752-bcf6-522ab61931fb" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Cp" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=69e5f764-c037-48d7-87a5-929bfa84f0c6 TargetServiceName=null EntryTimestamp=2021-08-11T16:16:07.852Z InvocationID=791b5687-5ba6-4792-8468-3ecd25a2bd0c ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:16:09.992Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Cp" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "69e5f764-c037-48d7-87a5-929bfa84f0c6" INVARIANT_UUID = "4566db8d-ca11-4752-bcf6-522ab61931fb" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Cp" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=69e5f764-c037-48d7-87a5-929bfa84f0c6 TargetServiceName=null EntryTimestamp=2021-08-11T16:16:07.852Z InvocationID=791b5687-5ba6-4792-8468-3ecd25a2bd0c ErrorCode=500 lock resource with id ca6bbeca-378f-4cd7-ad94-d713f62b8504 2021-08-11T16:16:10.690Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e603321-658b-4dcf-bbc3-59ab89e1b8ed PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:16:10.690Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e603321-658b-4dcf-bbc3-59ab89e1b8ed PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5e444566 2021-08-11T16:16:10.690Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e603321-658b-4dcf-bbc3-59ab89e1b8ed PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:16:10.690Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e603321-658b-4dcf-bbc3-59ab89e1b8ed PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:16:11.666Z [qtp2045766957-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Cp" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "69e5f764-c037-48d7-87a5-929bfa84f0c6" INVARIANT_UUID = "4566db8d-ca11-4752-bcf6-522ab61931fb" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "ca6bbeca-378f-4cd7-ad94-d713f62b8504.assettoscatemplate" CURR_ARTIFACT_UUID = "ca6bbeca-378f-4cd7-ad94-d713f62b8504.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Cp-template.yml,0,ca6bbeca-378f-4cd7-ad94-d713f62b8504.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=756241f7-c882-431d-960a-989dd38cbc33 TargetServiceName=null EntryTimestamp=2021-08-11T16:16:07.852Z InvocationID=791b5687-5ba6-4792-8468-3ecd25a2bd0c ErrorCode=500 Starting to create Csar for component Cp 2021-08-11T16:16:11.913Z [qtp2045766957-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Cp" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "69e5f764-c037-48d7-87a5-929bfa84f0c6" INVARIANT_UUID = "4566db8d-ca11-4752-bcf6-522ab61931fb" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "ca6bbeca-378f-4cd7-ad94-d713f62b8504.assettoscatemplate" CURR_ARTIFACT_UUID = "ca6bbeca-378f-4cd7-ad94-d713f62b8504.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Cp-template.yml,0,ca6bbeca-378f-4cd7-ad94-d713f62b8504.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=756241f7-c882-431d-960a-989dd38cbc33 TargetServiceName=null EntryTimestamp=2021-08-11T16:16:07.852Z InvocationID=791b5687-5ba6-4792-8468-3ecd25a2bd0c ErrorCode=500 Ended create Csar for component Cp 2021-08-11T16:16:11.913Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Cp" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "69e5f764-c037-48d7-87a5-929bfa84f0c6" INVARIANT_UUID = "4566db8d-ca11-4752-bcf6-522ab61931fb" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "ca6bbeca-378f-4cd7-ad94-d713f62b8504.assettoscatemplate" CURR_ARTIFACT_UUID = "ca6bbeca-378f-4cd7-ad94-d713f62b8504.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Cp-template.yml,0,ca6bbeca-378f-4cd7-ad94-d713f62b8504.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=756241f7-c882-431d-960a-989dd38cbc33 TargetServiceName=null EntryTimestamp=2021-08-11T16:16:07.852Z InvocationID=791b5687-5ba6-4792-8468-3ecd25a2bd0c ErrorCode=500 lock resource with id ca6bbeca-378f-4cd7-ad94-d713f62b8504 2021-08-11T16:16:13.592Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Cp" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "69e5f764-c037-48d7-87a5-929bfa84f0c6" INVARIANT_UUID = "4566db8d-ca11-4752-bcf6-522ab61931fb" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "ca6bbeca-378f-4cd7-ad94-d713f62b8504.assettoscacsar" CURR_ARTIFACT_UUID = "ca6bbeca-378f-4cd7-ad94-d713f62b8504.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Cp-csar.csar,0,ca6bbeca-378f-4cd7-ad94-d713f62b8504.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5739 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7b504406-4526-4414-a639-e3456996e27d TargetServiceName=null EntryTimestamp=2021-08-11T16:16:07.852Z InvocationID=791b5687-5ba6-4792-8468-3ecd25a2bd0c ErrorCode=500 LogTimestamp=2021-08-11T16:16:13.591984Z ResponseDesc=Created Exiting. 2021-08-11T16:16:13.592Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:16:13.609Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=f6991ffc-3395-4fe5-9474-83500f312373 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:16:13.609565Z InvocationID=0a25d933-1dac-4d1a-9378-0e8484de1c94 LogTimestamp=2021-08-11T16:16:13.609679Z Entering 2021-08-11T16:16:13.610Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=f6991ffc-3395-4fe5-9474-83500f312373 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:16:13.609565Z InvocationID=0a25d933-1dac-4d1a-9378-0e8484de1c94 LogTimestamp=2021-08-11T16:16:13.609679Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:16:13.693Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a0939de2-122c-4652-a710-67d252157925 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:16:13.693Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a0939de2-122c-4652-a710-67d252157925 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@504e2c10 2021-08-11T16:16:13.693Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a0939de2-122c-4652-a710-67d252157925 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:16:13.693Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a0939de2-122c-4652-a710-67d252157925 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:16:14.410Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:13.610Z InvocationID=3d70d74c-3f94-4fd0-86a4-f35ebe96fecf PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:14.411Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:13.610Z InvocationID=3d70d74c-3f94-4fd0-86a4-f35ebe96fecf PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:14.411Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:13.610Z InvocationID=3d70d74c-3f94-4fd0-86a4-f35ebe96fecf PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:14.411Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:13.610Z InvocationID=3d70d74c-3f94-4fd0-86a4-f35ebe96fecf PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:14.469Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:13.610Z InvocationID=3d70d74c-3f94-4fd0-86a4-f35ebe96fecf PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name VduVirtualstorage 2021-08-11T16:16:15.788Z [qtp2045766957-18] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "7c4ad5ad-ae15-4860-9efe-cb5963d7ce91" INVARIANT_UUID = "39f7e8ee-cd9f-482e-89c5-0509e063e0c7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualStorage" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7c4ad5ad-ae15-4860-9efe-cb5963d7ce91 EntryTimestamp=2021-08-11T16:16:13.610Z InvocationID=3d70d74c-3f94-4fd0-86a4-f35ebe96fecf PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:16:15.789Z [qtp2045766957-18] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "7c4ad5ad-ae15-4860-9efe-cb5963d7ce91" INVARIANT_UUID = "39f7e8ee-cd9f-482e-89c5-0509e063e0c7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7c4ad5ad-ae15-4860-9efe-cb5963d7ce91 TargetServiceName=null EntryTimestamp=2021-08-11T16:16:13.610Z InvocationID=3d70d74c-3f94-4fd0-86a4-f35ebe96fecf ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:16:16.032Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "7c4ad5ad-ae15-4860-9efe-cb5963d7ce91" INVARIANT_UUID = "39f7e8ee-cd9f-482e-89c5-0509e063e0c7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7c4ad5ad-ae15-4860-9efe-cb5963d7ce91 TargetServiceName=null EntryTimestamp=2021-08-11T16:16:13.610Z InvocationID=3d70d74c-3f94-4fd0-86a4-f35ebe96fecf ErrorCode=500 lock resource with id 27353e32-6ac6-42bd-9303-85fd90ddac32 2021-08-11T16:16:16.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70d9d87c-d114-4504-89d9-28577bf60d0f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:16:16.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70d9d87c-d114-4504-89d9-28577bf60d0f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@ceae280 2021-08-11T16:16:16.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70d9d87c-d114-4504-89d9-28577bf60d0f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:16:16.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70d9d87c-d114-4504-89d9-28577bf60d0f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:16:17.748Z [qtp2045766957-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "7c4ad5ad-ae15-4860-9efe-cb5963d7ce91" INVARIANT_UUID = "39f7e8ee-cd9f-482e-89c5-0509e063e0c7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "27353e32-6ac6-42bd-9303-85fd90ddac32.assettoscatemplate" CURR_ARTIFACT_UUID = "27353e32-6ac6-42bd-9303-85fd90ddac32.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualstorage-template.yml,0,27353e32-6ac6-42bd-9303-85fd90ddac32.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d860e689-a6dc-4447-8d6b-c7eb323d8de0 TargetServiceName=null EntryTimestamp=2021-08-11T16:16:13.610Z InvocationID=3d70d74c-3f94-4fd0-86a4-f35ebe96fecf ErrorCode=500 Starting to create Csar for component VDU VirtualStorage 2021-08-11T16:16:17.889Z [qtp2045766957-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "7c4ad5ad-ae15-4860-9efe-cb5963d7ce91" INVARIANT_UUID = "39f7e8ee-cd9f-482e-89c5-0509e063e0c7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "27353e32-6ac6-42bd-9303-85fd90ddac32.assettoscatemplate" CURR_ARTIFACT_UUID = "27353e32-6ac6-42bd-9303-85fd90ddac32.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualstorage-template.yml,0,27353e32-6ac6-42bd-9303-85fd90ddac32.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d860e689-a6dc-4447-8d6b-c7eb323d8de0 TargetServiceName=null EntryTimestamp=2021-08-11T16:16:13.610Z InvocationID=3d70d74c-3f94-4fd0-86a4-f35ebe96fecf ErrorCode=500 Ended create Csar for component VDU VirtualStorage 2021-08-11T16:16:17.890Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "7c4ad5ad-ae15-4860-9efe-cb5963d7ce91" INVARIANT_UUID = "39f7e8ee-cd9f-482e-89c5-0509e063e0c7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "27353e32-6ac6-42bd-9303-85fd90ddac32.assettoscatemplate" CURR_ARTIFACT_UUID = "27353e32-6ac6-42bd-9303-85fd90ddac32.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualstorage-template.yml,0,27353e32-6ac6-42bd-9303-85fd90ddac32.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d860e689-a6dc-4447-8d6b-c7eb323d8de0 TargetServiceName=null EntryTimestamp=2021-08-11T16:16:13.610Z InvocationID=3d70d74c-3f94-4fd0-86a4-f35ebe96fecf ErrorCode=500 lock resource with id 27353e32-6ac6-42bd-9303-85fd90ddac32 2021-08-11T16:16:18.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:16:18.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:16:19.592Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "7c4ad5ad-ae15-4860-9efe-cb5963d7ce91" INVARIANT_UUID = "39f7e8ee-cd9f-482e-89c5-0509e063e0c7" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "27353e32-6ac6-42bd-9303-85fd90ddac32.assettoscacsar" CURR_ARTIFACT_UUID = "27353e32-6ac6-42bd-9303-85fd90ddac32.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VduVirtualstorage-csar.csar,0,27353e32-6ac6-42bd-9303-85fd90ddac32.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5981 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7cb73f63-666a-42d9-bf8e-de5a68b3f889 TargetServiceName=null EntryTimestamp=2021-08-11T16:16:13.610Z InvocationID=3d70d74c-3f94-4fd0-86a4-f35ebe96fecf ErrorCode=500 LogTimestamp=2021-08-11T16:16:19.591905Z ResponseDesc=Created Exiting. 2021-08-11T16:16:19.592Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:16:19.653Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=cc2b24b7-6f18-44e5-bb13-ec7285f3e389 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:16:19.653119Z InvocationID=8ec77f74-e7fb-47a9-9224-7f8364459c30 LogTimestamp=2021-08-11T16:16:19.653178Z Entering 2021-08-11T16:16:19.654Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=cc2b24b7-6f18-44e5-bb13-ec7285f3e389 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:16:19.653119Z InvocationID=8ec77f74-e7fb-47a9-9224-7f8364459c30 LogTimestamp=2021-08-11T16:16:19.653178Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:16:20.050Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=005aa74c-b3da-462e-b8c2-d1bf714a0668 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:16:20.050Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=005aa74c-b3da-462e-b8c2-d1bf714a0668 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@582b4605 2021-08-11T16:16:20.050Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=005aa74c-b3da-462e-b8c2-d1bf714a0668 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:16:20.050Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=005aa74c-b3da-462e-b8c2-d1bf714a0668 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:16:20.624Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:19.654Z InvocationID=ea7391fb-a522-453f-afa3-32fcdb4044ca PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:20.624Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:19.654Z InvocationID=ea7391fb-a522-453f-afa3-32fcdb4044ca PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:20.624Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:19.654Z InvocationID=ea7391fb-a522-453f-afa3-32fcdb4044ca PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:20.624Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:19.654Z InvocationID=ea7391fb-a522-453f-afa3-32fcdb4044ca PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:20.640Z [qtp2045766957-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:19.654Z InvocationID=ea7391fb-a522-453f-afa3-32fcdb4044ca PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name VduVirtualblockstorage 2021-08-11T16:16:21.883Z [qtp2045766957-15] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "d8a810aa-d811-444f-863d-9a4d1134da1f" INVARIANT_UUID = "1a08d182-da17-4352-a1e3-b6174585e43f" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualBlockStorage" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d8a810aa-d811-444f-863d-9a4d1134da1f EntryTimestamp=2021-08-11T16:16:19.654Z InvocationID=ea7391fb-a522-453f-afa3-32fcdb4044ca PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:16:21.885Z [qtp2045766957-15] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "d8a810aa-d811-444f-863d-9a4d1134da1f" INVARIANT_UUID = "1a08d182-da17-4352-a1e3-b6174585e43f" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualBlockStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d8a810aa-d811-444f-863d-9a4d1134da1f TargetServiceName=null EntryTimestamp=2021-08-11T16:16:19.654Z InvocationID=ea7391fb-a522-453f-afa3-32fcdb4044ca ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:16:22.220Z [qtp2045766957-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "d8a810aa-d811-444f-863d-9a4d1134da1f" INVARIANT_UUID = "1a08d182-da17-4352-a1e3-b6174585e43f" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualBlockStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d8a810aa-d811-444f-863d-9a4d1134da1f TargetServiceName=null EntryTimestamp=2021-08-11T16:16:19.654Z InvocationID=ea7391fb-a522-453f-afa3-32fcdb4044ca ErrorCode=500 lock resource with id ad23e7cf-97bc-4368-981d-96424e77a3ee 2021-08-11T16:16:22.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8bb7c2cd-cc69-4ada-aabb-c02fcb667fe1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:16:22.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8bb7c2cd-cc69-4ada-aabb-c02fcb667fe1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@24b85713 2021-08-11T16:16:22.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8bb7c2cd-cc69-4ada-aabb-c02fcb667fe1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:16:22.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8bb7c2cd-cc69-4ada-aabb-c02fcb667fe1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:16:23.918Z [qtp2045766957-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "d8a810aa-d811-444f-863d-9a4d1134da1f" INVARIANT_UUID = "1a08d182-da17-4352-a1e3-b6174585e43f" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "ad23e7cf-97bc-4368-981d-96424e77a3ee.assettoscatemplate" CURR_ARTIFACT_UUID = "ad23e7cf-97bc-4368-981d-96424e77a3ee.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualblockstorage-template.yml,0,ad23e7cf-97bc-4368-981d-96424e77a3ee.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=32bed20e-6e60-4ab2-9d90-05081d5cdbbd TargetServiceName=null EntryTimestamp=2021-08-11T16:16:19.654Z InvocationID=ea7391fb-a522-453f-afa3-32fcdb4044ca ErrorCode=500 Starting to create Csar for component VDU VirtualBlockStorage 2021-08-11T16:16:24.041Z [qtp2045766957-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "d8a810aa-d811-444f-863d-9a4d1134da1f" INVARIANT_UUID = "1a08d182-da17-4352-a1e3-b6174585e43f" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "ad23e7cf-97bc-4368-981d-96424e77a3ee.assettoscatemplate" CURR_ARTIFACT_UUID = "ad23e7cf-97bc-4368-981d-96424e77a3ee.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualblockstorage-template.yml,0,ad23e7cf-97bc-4368-981d-96424e77a3ee.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=32bed20e-6e60-4ab2-9d90-05081d5cdbbd TargetServiceName=null EntryTimestamp=2021-08-11T16:16:19.654Z InvocationID=ea7391fb-a522-453f-afa3-32fcdb4044ca ErrorCode=500 Ended create Csar for component VDU VirtualBlockStorage 2021-08-11T16:16:24.042Z [qtp2045766957-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "d8a810aa-d811-444f-863d-9a4d1134da1f" INVARIANT_UUID = "1a08d182-da17-4352-a1e3-b6174585e43f" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "ad23e7cf-97bc-4368-981d-96424e77a3ee.assettoscatemplate" CURR_ARTIFACT_UUID = "ad23e7cf-97bc-4368-981d-96424e77a3ee.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualblockstorage-template.yml,0,ad23e7cf-97bc-4368-981d-96424e77a3ee.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=32bed20e-6e60-4ab2-9d90-05081d5cdbbd TargetServiceName=null EntryTimestamp=2021-08-11T16:16:19.654Z InvocationID=ea7391fb-a522-453f-afa3-32fcdb4044ca ErrorCode=500 lock resource with id ad23e7cf-97bc-4368-981d-96424e77a3ee 2021-08-11T16:16:25.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55812e2e-ef57-49d3-97e1-40d1ba844794 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:16:25.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55812e2e-ef57-49d3-97e1-40d1ba844794 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5b7c138 2021-08-11T16:16:25.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55812e2e-ef57-49d3-97e1-40d1ba844794 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:16:25.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55812e2e-ef57-49d3-97e1-40d1ba844794 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:16:25.762Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "d8a810aa-d811-444f-863d-9a4d1134da1f" INVARIANT_UUID = "1a08d182-da17-4352-a1e3-b6174585e43f" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "ad23e7cf-97bc-4368-981d-96424e77a3ee.assettoscacsar" CURR_ARTIFACT_UUID = "ad23e7cf-97bc-4368-981d-96424e77a3ee.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VduVirtualblockstorage-csar.csar,0,ad23e7cf-97bc-4368-981d-96424e77a3ee.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=6107 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9b7d6e3d-9015-488f-b110-f8042c59d101 TargetServiceName=null EntryTimestamp=2021-08-11T16:16:19.654Z InvocationID=ea7391fb-a522-453f-afa3-32fcdb4044ca ErrorCode=500 LogTimestamp=2021-08-11T16:16:25.761560Z ResponseDesc=Created Exiting. 2021-08-11T16:16:25.763Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:16:25.783Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=d6d409b2-9851-4249-b2e5-b6a3bfd139dd InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:16:25.783209Z InvocationID=3e64eb46-10e2-4408-875a-55076fd46e38 LogTimestamp=2021-08-11T16:16:25.783256Z Entering 2021-08-11T16:16:25.783Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=d6d409b2-9851-4249-b2e5-b6a3bfd139dd InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:16:25.783209Z InvocationID=3e64eb46-10e2-4408-875a-55076fd46e38 LogTimestamp=2021-08-11T16:16:25.783256Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:16:26.949Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:25.783Z InvocationID=780b3c86-42f7-4e31-951a-6afeafe6a5a6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:26.949Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:25.783Z InvocationID=780b3c86-42f7-4e31-951a-6afeafe6a5a6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:26.949Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:25.783Z InvocationID=780b3c86-42f7-4e31-951a-6afeafe6a5a6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:26.949Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:25.783Z InvocationID=780b3c86-42f7-4e31-951a-6afeafe6a5a6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:27.061Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:25.783Z InvocationID=780b3c86-42f7-4e31-951a-6afeafe6a5a6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name VduVirtualfilestorage 2021-08-11T16:16:28.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=030d28d1-d628-4c0b-a79d-17e16ca2787e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:16:28.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=030d28d1-d628-4c0b-a79d-17e16ca2787e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5e12cdec 2021-08-11T16:16:28.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=030d28d1-d628-4c0b-a79d-17e16ca2787e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:16:28.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=030d28d1-d628-4c0b-a79d-17e16ca2787e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:16:29.153Z [qtp2045766957-18] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "79702974-9be4-4609-9623-ba3dcd93a111" INVARIANT_UUID = "6d5e9aee-a62c-4f59-a5bb-2bc9216796e4" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualFileStorage" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=79702974-9be4-4609-9623-ba3dcd93a111 EntryTimestamp=2021-08-11T16:16:25.783Z InvocationID=780b3c86-42f7-4e31-951a-6afeafe6a5a6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:16:29.154Z [qtp2045766957-18] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "79702974-9be4-4609-9623-ba3dcd93a111" INVARIANT_UUID = "6d5e9aee-a62c-4f59-a5bb-2bc9216796e4" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualFileStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=79702974-9be4-4609-9623-ba3dcd93a111 TargetServiceName=null EntryTimestamp=2021-08-11T16:16:25.783Z InvocationID=780b3c86-42f7-4e31-951a-6afeafe6a5a6 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:16:30.058Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "79702974-9be4-4609-9623-ba3dcd93a111" INVARIANT_UUID = "6d5e9aee-a62c-4f59-a5bb-2bc9216796e4" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualFileStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=79702974-9be4-4609-9623-ba3dcd93a111 TargetServiceName=null EntryTimestamp=2021-08-11T16:16:25.783Z InvocationID=780b3c86-42f7-4e31-951a-6afeafe6a5a6 ErrorCode=500 lock resource with id 0bd866ed-f620-48c3-889c-5bff0971b4f2 2021-08-11T16:16:31.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4307d21b-e373-452f-89ff-356934fe75c8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:16:31.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4307d21b-e373-452f-89ff-356934fe75c8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5c0ba345 2021-08-11T16:16:31.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4307d21b-e373-452f-89ff-356934fe75c8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:16:31.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4307d21b-e373-452f-89ff-356934fe75c8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:16:31.698Z [qtp2045766957-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "79702974-9be4-4609-9623-ba3dcd93a111" INVARIANT_UUID = "6d5e9aee-a62c-4f59-a5bb-2bc9216796e4" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "0bd866ed-f620-48c3-889c-5bff0971b4f2.assettoscatemplate" CURR_ARTIFACT_UUID = "0bd866ed-f620-48c3-889c-5bff0971b4f2.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualfilestorage-template.yml,0,0bd866ed-f620-48c3-889c-5bff0971b4f2.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=15cc45d1-d4ca-4ed6-b73d-12d82ac2bb83 TargetServiceName=null EntryTimestamp=2021-08-11T16:16:25.783Z InvocationID=780b3c86-42f7-4e31-951a-6afeafe6a5a6 ErrorCode=500 Starting to create Csar for component VDU VirtualFileStorage 2021-08-11T16:16:31.871Z [qtp2045766957-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "79702974-9be4-4609-9623-ba3dcd93a111" INVARIANT_UUID = "6d5e9aee-a62c-4f59-a5bb-2bc9216796e4" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "0bd866ed-f620-48c3-889c-5bff0971b4f2.assettoscatemplate" CURR_ARTIFACT_UUID = "0bd866ed-f620-48c3-889c-5bff0971b4f2.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualfilestorage-template.yml,0,0bd866ed-f620-48c3-889c-5bff0971b4f2.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=15cc45d1-d4ca-4ed6-b73d-12d82ac2bb83 TargetServiceName=null EntryTimestamp=2021-08-11T16:16:25.783Z InvocationID=780b3c86-42f7-4e31-951a-6afeafe6a5a6 ErrorCode=500 Ended create Csar for component VDU VirtualFileStorage 2021-08-11T16:16:31.872Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "79702974-9be4-4609-9623-ba3dcd93a111" INVARIANT_UUID = "6d5e9aee-a62c-4f59-a5bb-2bc9216796e4" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "0bd866ed-f620-48c3-889c-5bff0971b4f2.assettoscatemplate" CURR_ARTIFACT_UUID = "0bd866ed-f620-48c3-889c-5bff0971b4f2.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualfilestorage-template.yml,0,0bd866ed-f620-48c3-889c-5bff0971b4f2.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=15cc45d1-d4ca-4ed6-b73d-12d82ac2bb83 TargetServiceName=null EntryTimestamp=2021-08-11T16:16:25.783Z InvocationID=780b3c86-42f7-4e31-951a-6afeafe6a5a6 ErrorCode=500 lock resource with id 0bd866ed-f620-48c3-889c-5bff0971b4f2 2021-08-11T16:16:33.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:16:33.317Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:16:33.548Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "79702974-9be4-4609-9623-ba3dcd93a111" INVARIANT_UUID = "6d5e9aee-a62c-4f59-a5bb-2bc9216796e4" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "0bd866ed-f620-48c3-889c-5bff0971b4f2.assettoscacsar" CURR_ARTIFACT_UUID = "0bd866ed-f620-48c3-889c-5bff0971b4f2.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VduVirtualfilestorage-csar.csar,0,0bd866ed-f620-48c3-889c-5bff0971b4f2.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=7764 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=4e14a40c-b70c-4c85-b5e5-ba3b14955d1b TargetServiceName=null EntryTimestamp=2021-08-11T16:16:25.783Z InvocationID=780b3c86-42f7-4e31-951a-6afeafe6a5a6 ErrorCode=500 LogTimestamp=2021-08-11T16:16:33.547851Z ResponseDesc=Created Exiting. 2021-08-11T16:16:33.549Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:16:33.570Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=29ea32c1-a34c-4d54-a917-9ee3094a6adf InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:16:33.570441Z InvocationID=a1754e50-6492-439b-ac3b-b3b8f781acc3 LogTimestamp=2021-08-11T16:16:33.570480Z Entering 2021-08-11T16:16:33.570Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=29ea32c1-a34c-4d54-a917-9ee3094a6adf InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:16:33.570441Z InvocationID=a1754e50-6492-439b-ac3b-b3b8f781acc3 LogTimestamp=2021-08-11T16:16:33.570480Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:16:33.870Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:33.571Z InvocationID=8d995299-a578-46c7-b663-30bef4823e84 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:33.870Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:33.571Z InvocationID=8d995299-a578-46c7-b663-30bef4823e84 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:33.870Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:33.571Z InvocationID=8d995299-a578-46c7-b663-30bef4823e84 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:33.870Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:33.571Z InvocationID=8d995299-a578-46c7-b663-30bef4823e84 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:33.882Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:33.571Z InvocationID=8d995299-a578-46c7-b663-30bef4823e84 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name VduVirtualobjectstorage 2021-08-11T16:16:34.715Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c08101bd-327a-4390-b8fa-15ef2437b7ea PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:16:34.715Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c08101bd-327a-4390-b8fa-15ef2437b7ea PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3a6d7718 2021-08-11T16:16:34.715Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c08101bd-327a-4390-b8fa-15ef2437b7ea PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:16:34.715Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c08101bd-327a-4390-b8fa-15ef2437b7ea PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:16:35.364Z [qtp2045766957-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "49d73609-3857-4467-b56e-5a12862471ea" INVARIANT_UUID = "2000592c-7dbc-4d8f-b39f-eec970daca32" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualObjectStorage" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=49d73609-3857-4467-b56e-5a12862471ea EntryTimestamp=2021-08-11T16:16:33.571Z InvocationID=8d995299-a578-46c7-b663-30bef4823e84 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:16:35.365Z [qtp2045766957-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "49d73609-3857-4467-b56e-5a12862471ea" INVARIANT_UUID = "2000592c-7dbc-4d8f-b39f-eec970daca32" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualObjectStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=49d73609-3857-4467-b56e-5a12862471ea TargetServiceName=null EntryTimestamp=2021-08-11T16:16:33.571Z InvocationID=8d995299-a578-46c7-b663-30bef4823e84 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:16:35.730Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "49d73609-3857-4467-b56e-5a12862471ea" INVARIANT_UUID = "2000592c-7dbc-4d8f-b39f-eec970daca32" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualObjectStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=49d73609-3857-4467-b56e-5a12862471ea TargetServiceName=null EntryTimestamp=2021-08-11T16:16:33.571Z InvocationID=8d995299-a578-46c7-b663-30bef4823e84 ErrorCode=500 lock resource with id 123fd3d9-f85b-4457-a72b-93bac84ac22c 2021-08-11T16:16:37.355Z [qtp2045766957-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "49d73609-3857-4467-b56e-5a12862471ea" INVARIANT_UUID = "2000592c-7dbc-4d8f-b39f-eec970daca32" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "123fd3d9-f85b-4457-a72b-93bac84ac22c.assettoscatemplate" CURR_ARTIFACT_UUID = "123fd3d9-f85b-4457-a72b-93bac84ac22c.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualobjectstorage-template.yml,0,123fd3d9-f85b-4457-a72b-93bac84ac22c.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9c042db2-0315-43d4-bf61-f17ecd584586 TargetServiceName=null EntryTimestamp=2021-08-11T16:16:33.571Z InvocationID=8d995299-a578-46c7-b663-30bef4823e84 ErrorCode=500 Starting to create Csar for component VDU VirtualObjectStorage 2021-08-11T16:16:37.553Z [qtp2045766957-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "49d73609-3857-4467-b56e-5a12862471ea" INVARIANT_UUID = "2000592c-7dbc-4d8f-b39f-eec970daca32" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "123fd3d9-f85b-4457-a72b-93bac84ac22c.assettoscatemplate" CURR_ARTIFACT_UUID = "123fd3d9-f85b-4457-a72b-93bac84ac22c.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualobjectstorage-template.yml,0,123fd3d9-f85b-4457-a72b-93bac84ac22c.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9c042db2-0315-43d4-bf61-f17ecd584586 TargetServiceName=null EntryTimestamp=2021-08-11T16:16:33.571Z InvocationID=8d995299-a578-46c7-b663-30bef4823e84 ErrorCode=500 Ended create Csar for component VDU VirtualObjectStorage 2021-08-11T16:16:37.554Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "49d73609-3857-4467-b56e-5a12862471ea" INVARIANT_UUID = "2000592c-7dbc-4d8f-b39f-eec970daca32" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "123fd3d9-f85b-4457-a72b-93bac84ac22c.assettoscatemplate" CURR_ARTIFACT_UUID = "123fd3d9-f85b-4457-a72b-93bac84ac22c.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualobjectstorage-template.yml,0,123fd3d9-f85b-4457-a72b-93bac84ac22c.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9c042db2-0315-43d4-bf61-f17ecd584586 TargetServiceName=null EntryTimestamp=2021-08-11T16:16:33.571Z InvocationID=8d995299-a578-46c7-b663-30bef4823e84 ErrorCode=500 lock resource with id 123fd3d9-f85b-4457-a72b-93bac84ac22c 2021-08-11T16:16:37.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5eaa6550-d098-49f6-b2f8-dac7702f8cc7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:16:37.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5eaa6550-d098-49f6-b2f8-dac7702f8cc7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1e8df6c 2021-08-11T16:16:37.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5eaa6550-d098-49f6-b2f8-dac7702f8cc7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:16:37.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5eaa6550-d098-49f6-b2f8-dac7702f8cc7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:16:39.189Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualObjectStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "49d73609-3857-4467-b56e-5a12862471ea" INVARIANT_UUID = "2000592c-7dbc-4d8f-b39f-eec970daca32" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "123fd3d9-f85b-4457-a72b-93bac84ac22c.assettoscacsar" CURR_ARTIFACT_UUID = "123fd3d9-f85b-4457-a72b-93bac84ac22c.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VduVirtualobjectstorage-csar.csar,0,123fd3d9-f85b-4457-a72b-93bac84ac22c.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5618 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=559a9029-75e9-476b-b175-df89dd6663dd TargetServiceName=null EntryTimestamp=2021-08-11T16:16:33.571Z InvocationID=8d995299-a578-46c7-b663-30bef4823e84 ErrorCode=500 LogTimestamp=2021-08-11T16:16:39.189133Z ResponseDesc=Created Exiting. 2021-08-11T16:16:39.190Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:16:39.210Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=789b67d1-d69b-4203-b8bd-bec313a9119e InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:16:39.209922Z InvocationID=92964b79-761e-43e6-9d23-8f8323cb9175 LogTimestamp=2021-08-11T16:16:39.209968Z Entering 2021-08-11T16:16:39.210Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=789b67d1-d69b-4203-b8bd-bec313a9119e InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:16:39.209922Z InvocationID=92964b79-761e-43e6-9d23-8f8323cb9175 LogTimestamp=2021-08-11T16:16:39.209968Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:16:39.498Z [qtp2045766957-18] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=Import Resource vduVirtualStorage.yml ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2021-08-11T16:16:39.210Z InvocationID=5fb50013-c165-44a2-8793-6a934a5915c0 ErrorCode=500 Unexpected error during operation 2021-08-11T16:16:39.502Z [qtp2045766957-18] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "" INVARIANT_UUID = "" PREV_VERSION = "1.0" CURR_VERSION = "" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "" STATUS = "409" DESC = "SVC4083: Error: Version of this VDU VirtualStorage was already promoted." TOSCA_NODE_TYPE = "" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=Upload Resource ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2021-08-11T16:16:39.210Z InvocationID=5fb50013-c165-44a2-8793-6a934a5915c0 ErrorCode=900 Unexpected error during BE REST API execution 2021-08-11T16:16:39.504Z [qtp2045766957-18] ERROR o.o.s.b.s.ResourceUploadServlet AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "" INVARIANT_UUID = "" PREV_VERSION = "1.0" CURR_VERSION = "" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "" STATUS = "409" DESC = "SVC4083: Error: Version of this VDU VirtualStorage was already promoted." TOSCA_NODE_TYPE = "" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=Upload Resource ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2021-08-11T16:16:39.210Z InvocationID=5fb50013-c165-44a2-8793-6a934a5915c0 ErrorCode=900 #uploadMultipart('multipart', /tmp/rep740026284248358758tmp, 'form-data; filename="vduVirtualStorage.zip"; name="resourceZip"', '{"payloadName": "vduVirtualStorage.yml", "contactId": "jh0003", "name": "VDU VirtualStorage", "description": "VDU VirtualStorage", "resourceIconPath": "objectStorage", "resourceType": "VFC", "categories": [{"name": "Generic", "subcategories": [{"name": "Infrastructure"}]}], "tags": ["VDU VirtualStorage"]}', 'Request(POST https://sdc-be.onap:8443/sdc2/rest/v1/catalog/upload/multipart?createNewVersion=true)@52a4e9a9', 'jh0003', true): thrown org.openecomp.sdc.be.components.impl.exceptions.ByActionStatusComponentException out of org.openecomp.sdc.be.components.impl.ResourceImportManager#importCertifiedResource[184] in 290ms 2021-08-11T16:16:39.506Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "" INVARIANT_UUID = "" PREV_VERSION = "1.0" CURR_VERSION = "" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "" STATUS = "409" DESC = "SVC4083: Error: Version of this VDU VirtualStorage was already promoted." TOSCA_NODE_TYPE = "" ResponseCode=409 RequestID=null ElapsedTime=295 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=ERROR TargetEntity=Upload Resource ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null ErrorDesc=Conflict EntryTimestamp=2021-08-11T16:16:39.210Z InvocationID=5fb50013-c165-44a2-8793-6a934a5915c0 ErrorCode=409 LogTimestamp=2021-08-11T16:16:39.505950Z ResponseDesc=Conflict Exiting. 2021-08-11T16:16:39.507Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="409" 2021-08-11T16:16:39.521Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=56a69aa1-fa52-466c-af9c-e57da7d87839 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:16:39.521Z InvocationID=981846d1-60ba-49bb-b5bb-81fc29b24034 LogTimestamp=2021-08-11T16:16:39.521076Z Entering 2021-08-11T16:16:39.521Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=56a69aa1-fa52-466c-af9c-e57da7d87839 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:16:39.521Z InvocationID=981846d1-60ba-49bb-b5bb-81fc29b24034 LogTimestamp=2021-08-11T16:16:39.521076Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:16:39.875Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:39.521Z InvocationID=2fd27a2a-d25d-48aa-8bde-b8cc4f3ee34c PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:39.875Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:39.521Z InvocationID=2fd27a2a-d25d-48aa-8bde-b8cc4f3ee34c PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:39.875Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:39.521Z InvocationID=2fd27a2a-d25d-48aa-8bde-b8cc4f3ee34c PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:39.875Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:39.521Z InvocationID=2fd27a2a-d25d-48aa-8bde-b8cc4f3ee34c PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:39.886Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:39.521Z InvocationID=2fd27a2a-d25d-48aa-8bde-b8cc4f3ee34c PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Vnfvirtuallink 2021-08-11T16:16:40.758Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3bbe631-27bd-41ad-a5ee-a114953d118f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:16:40.758Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3bbe631-27bd-41ad-a5ee-a114953d118f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@49dd1f10 2021-08-11T16:16:40.758Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3bbe631-27bd-41ad-a5ee-a114953d118f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:16:40.758Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a3bbe631-27bd-41ad-a5ee-a114953d118f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:16:41.269Z [qtp2045766957-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VnfVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "caea52b4-5ac9-4749-ac5d-1e7b61451f96" INVARIANT_UUID = "51cb4ed9-6777-4e5f-ba36-a74fd233e696" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VnfVirtualLink" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=caea52b4-5ac9-4749-ac5d-1e7b61451f96 EntryTimestamp=2021-08-11T16:16:39.521Z InvocationID=2fd27a2a-d25d-48aa-8bde-b8cc4f3ee34c PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:16:41.270Z [qtp2045766957-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VnfVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "caea52b4-5ac9-4749-ac5d-1e7b61451f96" INVARIANT_UUID = "51cb4ed9-6777-4e5f-ba36-a74fd233e696" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VnfVirtualLink" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=caea52b4-5ac9-4749-ac5d-1e7b61451f96 TargetServiceName=null EntryTimestamp=2021-08-11T16:16:39.521Z InvocationID=2fd27a2a-d25d-48aa-8bde-b8cc4f3ee34c ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:16:41.554Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VnfVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "caea52b4-5ac9-4749-ac5d-1e7b61451f96" INVARIANT_UUID = "51cb4ed9-6777-4e5f-ba36-a74fd233e696" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VnfVirtualLink" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=caea52b4-5ac9-4749-ac5d-1e7b61451f96 TargetServiceName=null EntryTimestamp=2021-08-11T16:16:39.521Z InvocationID=2fd27a2a-d25d-48aa-8bde-b8cc4f3ee34c ErrorCode=500 lock resource with id 54597895-ea1d-41d7-baec-364d29715d04 2021-08-11T16:16:43.178Z [qtp2045766957-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VnfVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "caea52b4-5ac9-4749-ac5d-1e7b61451f96" INVARIANT_UUID = "51cb4ed9-6777-4e5f-ba36-a74fd233e696" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "54597895-ea1d-41d7-baec-364d29715d04.assettoscatemplate" CURR_ARTIFACT_UUID = "54597895-ea1d-41d7-baec-364d29715d04.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vnfvirtuallink-template.yml,0,54597895-ea1d-41d7-baec-364d29715d04.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b7e00fbf-282a-40a4-8c4f-55ae217572b1 TargetServiceName=null EntryTimestamp=2021-08-11T16:16:39.521Z InvocationID=2fd27a2a-d25d-48aa-8bde-b8cc4f3ee34c ErrorCode=500 Starting to create Csar for component VnfVirtualLink 2021-08-11T16:16:43.362Z [qtp2045766957-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VnfVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "caea52b4-5ac9-4749-ac5d-1e7b61451f96" INVARIANT_UUID = "51cb4ed9-6777-4e5f-ba36-a74fd233e696" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "54597895-ea1d-41d7-baec-364d29715d04.assettoscatemplate" CURR_ARTIFACT_UUID = "54597895-ea1d-41d7-baec-364d29715d04.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vnfvirtuallink-template.yml,0,54597895-ea1d-41d7-baec-364d29715d04.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b7e00fbf-282a-40a4-8c4f-55ae217572b1 TargetServiceName=null EntryTimestamp=2021-08-11T16:16:39.521Z InvocationID=2fd27a2a-d25d-48aa-8bde-b8cc4f3ee34c ErrorCode=500 Ended create Csar for component VnfVirtualLink 2021-08-11T16:16:43.362Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VnfVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "caea52b4-5ac9-4749-ac5d-1e7b61451f96" INVARIANT_UUID = "51cb4ed9-6777-4e5f-ba36-a74fd233e696" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "54597895-ea1d-41d7-baec-364d29715d04.assettoscatemplate" CURR_ARTIFACT_UUID = "54597895-ea1d-41d7-baec-364d29715d04.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vnfvirtuallink-template.yml,0,54597895-ea1d-41d7-baec-364d29715d04.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b7e00fbf-282a-40a4-8c4f-55ae217572b1 TargetServiceName=null EntryTimestamp=2021-08-11T16:16:39.521Z InvocationID=2fd27a2a-d25d-48aa-8bde-b8cc4f3ee34c ErrorCode=500 lock resource with id 54597895-ea1d-41d7-baec-364d29715d04 2021-08-11T16:16:43.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8cbd6db4-3d59-4d52-b6fd-098c00811cf4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:16:43.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8cbd6db4-3d59-4d52-b6fd-098c00811cf4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@484d9ce5 2021-08-11T16:16:43.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8cbd6db4-3d59-4d52-b6fd-098c00811cf4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:16:43.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8cbd6db4-3d59-4d52-b6fd-098c00811cf4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:16:45.009Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VnfVirtualLink" RESOURCE_TYPE = "VL" SERVICE_INSTANCE_ID = "caea52b4-5ac9-4749-ac5d-1e7b61451f96" INVARIANT_UUID = "51cb4ed9-6777-4e5f-ba36-a74fd233e696" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "54597895-ea1d-41d7-baec-364d29715d04.assettoscacsar" CURR_ARTIFACT_UUID = "54597895-ea1d-41d7-baec-364d29715d04.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Vnfvirtuallink-csar.csar,0,54597895-ea1d-41d7-baec-364d29715d04.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5487 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c5c9c1c4-cdd9-4711-a103-9a2c868496a3 TargetServiceName=null EntryTimestamp=2021-08-11T16:16:39.521Z InvocationID=2fd27a2a-d25d-48aa-8bde-b8cc4f3ee34c ErrorCode=500 LogTimestamp=2021-08-11T16:16:45.008476Z ResponseDesc=Created Exiting. 2021-08-11T16:16:45.009Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:16:45.031Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=36a560fb-d6e6-4822-b50a-00182cd35686 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:16:45.030933Z InvocationID=c3f8ff75-a0f1-43f4-a94e-535704d678eb LogTimestamp=2021-08-11T16:16:45.030981Z Entering 2021-08-11T16:16:45.031Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=36a560fb-d6e6-4822-b50a-00182cd35686 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:16:45.030933Z InvocationID=c3f8ff75-a0f1-43f4-a94e-535704d678eb LogTimestamp=2021-08-11T16:16:45.030981Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:16:45.377Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:45.031Z InvocationID=2da3f6c7-09ef-4601-a16c-24e3eb26887b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:45.377Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:45.031Z InvocationID=2da3f6c7-09ef-4601-a16c-24e3eb26887b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:45.377Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:45.031Z InvocationID=2da3f6c7-09ef-4601-a16c-24e3eb26887b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:45.377Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:45.031Z InvocationID=2da3f6c7-09ef-4601-a16c-24e3eb26887b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:45.392Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:45.031Z InvocationID=2da3f6c7-09ef-4601-a16c-24e3eb26887b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name VnfExternalCp 2021-08-11T16:16:46.628Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=866807c0-a152-4d25-81d1-730cf1b7e50f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:16:46.628Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=866807c0-a152-4d25-81d1-730cf1b7e50f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1949f743 2021-08-11T16:16:46.628Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=866807c0-a152-4d25-81d1-730cf1b7e50f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:16:46.628Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=866807c0-a152-4d25-81d1-730cf1b7e50f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:16:46.888Z [qtp2045766957-12] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "6d5451c9-5f60-4a28-ab1a-ddc08edf4df7" INVARIANT_UUID = "5fce6f56-b8f4-4441-a8d6-1217f0721dfe" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VnfExtCp" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=6d5451c9-5f60-4a28-ab1a-ddc08edf4df7 EntryTimestamp=2021-08-11T16:16:45.031Z InvocationID=2da3f6c7-09ef-4601-a16c-24e3eb26887b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:16:46.889Z [qtp2045766957-12] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "6d5451c9-5f60-4a28-ab1a-ddc08edf4df7" INVARIANT_UUID = "5fce6f56-b8f4-4441-a8d6-1217f0721dfe" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VnfExtCp" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=6d5451c9-5f60-4a28-ab1a-ddc08edf4df7 TargetServiceName=null EntryTimestamp=2021-08-11T16:16:45.031Z InvocationID=2da3f6c7-09ef-4601-a16c-24e3eb26887b ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:16:47.131Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "6d5451c9-5f60-4a28-ab1a-ddc08edf4df7" INVARIANT_UUID = "5fce6f56-b8f4-4441-a8d6-1217f0721dfe" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VnfExtCp" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=6d5451c9-5f60-4a28-ab1a-ddc08edf4df7 TargetServiceName=null EntryTimestamp=2021-08-11T16:16:45.031Z InvocationID=2da3f6c7-09ef-4601-a16c-24e3eb26887b ErrorCode=500 lock resource with id cf9121f9-869a-42a4-a3b3-66aff5eeb3a3 2021-08-11T16:16:48.309Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:16:48.316Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:16:48.769Z [qtp2045766957-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "6d5451c9-5f60-4a28-ab1a-ddc08edf4df7" INVARIANT_UUID = "5fce6f56-b8f4-4441-a8d6-1217f0721dfe" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "cf9121f9-869a-42a4-a3b3-66aff5eeb3a3.assettoscatemplate" CURR_ARTIFACT_UUID = "cf9121f9-869a-42a4-a3b3-66aff5eeb3a3.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VnfExternalCp-template.yml,0,cf9121f9-869a-42a4-a3b3-66aff5eeb3a3.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f590acd1-c098-45b1-839a-3a973f1181b4 TargetServiceName=null EntryTimestamp=2021-08-11T16:16:45.031Z InvocationID=2da3f6c7-09ef-4601-a16c-24e3eb26887b ErrorCode=500 Starting to create Csar for component VNF External CP 2021-08-11T16:16:48.931Z [qtp2045766957-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "6d5451c9-5f60-4a28-ab1a-ddc08edf4df7" INVARIANT_UUID = "5fce6f56-b8f4-4441-a8d6-1217f0721dfe" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "cf9121f9-869a-42a4-a3b3-66aff5eeb3a3.assettoscatemplate" CURR_ARTIFACT_UUID = "cf9121f9-869a-42a4-a3b3-66aff5eeb3a3.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VnfExternalCp-template.yml,0,cf9121f9-869a-42a4-a3b3-66aff5eeb3a3.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f590acd1-c098-45b1-839a-3a973f1181b4 TargetServiceName=null EntryTimestamp=2021-08-11T16:16:45.031Z InvocationID=2da3f6c7-09ef-4601-a16c-24e3eb26887b ErrorCode=500 Ended create Csar for component VNF External CP 2021-08-11T16:16:48.932Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "6d5451c9-5f60-4a28-ab1a-ddc08edf4df7" INVARIANT_UUID = "5fce6f56-b8f4-4441-a8d6-1217f0721dfe" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "cf9121f9-869a-42a4-a3b3-66aff5eeb3a3.assettoscatemplate" CURR_ARTIFACT_UUID = "cf9121f9-869a-42a4-a3b3-66aff5eeb3a3.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VnfExternalCp-template.yml,0,cf9121f9-869a-42a4-a3b3-66aff5eeb3a3.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f590acd1-c098-45b1-839a-3a973f1181b4 TargetServiceName=null EntryTimestamp=2021-08-11T16:16:45.031Z InvocationID=2da3f6c7-09ef-4601-a16c-24e3eb26887b ErrorCode=500 lock resource with id cf9121f9-869a-42a4-a3b3-66aff5eeb3a3 2021-08-11T16:16:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:16:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:16:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:16:50.357Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb7d2b4d-7491-465e-b25d-1e18484ea731 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:16:50.358Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb7d2b4d-7491-465e-b25d-1e18484ea731 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5e9d48b9 2021-08-11T16:16:50.358Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb7d2b4d-7491-465e-b25d-1e18484ea731 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:16:50.358Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb7d2b4d-7491-465e-b25d-1e18484ea731 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:16:51.262Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "6d5451c9-5f60-4a28-ab1a-ddc08edf4df7" INVARIANT_UUID = "5fce6f56-b8f4-4441-a8d6-1217f0721dfe" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "cf9121f9-869a-42a4-a3b3-66aff5eeb3a3.assettoscacsar" CURR_ARTIFACT_UUID = "cf9121f9-869a-42a4-a3b3-66aff5eeb3a3.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VnfExternalCp-csar.csar,0,cf9121f9-869a-42a4-a3b3-66aff5eeb3a3.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=6231 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=74c9b262-551d-4361-a84d-4f41986567b3 TargetServiceName=null EntryTimestamp=2021-08-11T16:16:45.031Z InvocationID=2da3f6c7-09ef-4601-a16c-24e3eb26887b ErrorCode=500 LogTimestamp=2021-08-11T16:16:51.262358Z ResponseDesc=Created Exiting. 2021-08-11T16:16:51.263Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:16:51.366Z [qtp2045766957-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=333f2e4a-dc36-4bf1-8546-aa9af6a5cb20 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:16:51.366312Z InvocationID=52daf20a-ebea-4284-816e-ef52e53427f3 LogTimestamp=2021-08-11T16:16:51.366378Z Entering 2021-08-11T16:16:51.366Z [qtp2045766957-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=333f2e4a-dc36-4bf1-8546-aa9af6a5cb20 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:16:51.366312Z InvocationID=52daf20a-ebea-4284-816e-ef52e53427f3 LogTimestamp=2021-08-11T16:16:51.366378Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:16:53.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab9318a9-33c5-43ee-8a18-81093bd50e3d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:16:53.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab9318a9-33c5-43ee-8a18-81093bd50e3d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7f99f9e5 2021-08-11T16:16:53.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab9318a9-33c5-43ee-8a18-81093bd50e3d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:16:53.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab9318a9-33c5-43ee-8a18-81093bd50e3d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:16:54.665Z [qtp2045766957-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:51.367Z InvocationID=6d66ba2b-a44a-407d-a8f7-0aa6814b5e8e PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:54.665Z [qtp2045766957-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:51.367Z InvocationID=6d66ba2b-a44a-407d-a8f7-0aa6814b5e8e PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:54.666Z [qtp2045766957-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:51.367Z InvocationID=6d66ba2b-a44a-407d-a8f7-0aa6814b5e8e PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:54.666Z [qtp2045766957-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:51.367Z InvocationID=6d66ba2b-a44a-407d-a8f7-0aa6814b5e8e PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:16:54.866Z [qtp2045766957-20] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:16:51.367Z InvocationID=6d66ba2b-a44a-407d-a8f7-0aa6814b5e8e PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name VduCp 2021-08-11T16:16:57.154Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd3b74c5-f085-42cd-89e1-efc51215bbf2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:16:57.154Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd3b74c5-f085-42cd-89e1-efc51215bbf2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5b17e847 2021-08-11T16:16:57.154Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd3b74c5-f085-42cd-89e1-efc51215bbf2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:16:57.154Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd3b74c5-f085-42cd-89e1-efc51215bbf2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:16:58.949Z [qtp2045766957-20] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "3fb43b0c-237f-49b9-8515-4c723378e4df" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VduCp" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3fb43b0c-237f-49b9-8515-4c723378e4df EntryTimestamp=2021-08-11T16:16:51.367Z InvocationID=6d66ba2b-a44a-407d-a8f7-0aa6814b5e8e PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:16:58.950Z [qtp2045766957-20] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "3fb43b0c-237f-49b9-8515-4c723378e4df" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VduCp" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3fb43b0c-237f-49b9-8515-4c723378e4df TargetServiceName=null EntryTimestamp=2021-08-11T16:16:51.367Z InvocationID=6d66ba2b-a44a-407d-a8f7-0aa6814b5e8e ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:17:00.373Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77903259-2b47-4fc7-8bdb-54788b5b1f97 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:17:00.373Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77903259-2b47-4fc7-8bdb-54788b5b1f97 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@d32ff89 2021-08-11T16:17:00.373Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77903259-2b47-4fc7-8bdb-54788b5b1f97 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:17:00.373Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77903259-2b47-4fc7-8bdb-54788b5b1f97 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:17:01.651Z [qtp2045766957-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "3fb43b0c-237f-49b9-8515-4c723378e4df" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VduCp" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3fb43b0c-237f-49b9-8515-4c723378e4df TargetServiceName=null EntryTimestamp=2021-08-11T16:16:51.367Z InvocationID=6d66ba2b-a44a-407d-a8f7-0aa6814b5e8e ErrorCode=500 lock resource with id 244ea341-a6db-4cce-8507-7871b5ec5d3d 2021-08-11T16:17:03.067Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f9103344-fd6f-4302-9873-a95b27d1d363 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:17:03.067Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f9103344-fd6f-4302-9873-a95b27d1d363 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@395eb616 2021-08-11T16:17:03.067Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f9103344-fd6f-4302-9873-a95b27d1d363 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:17:03.067Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f9103344-fd6f-4302-9873-a95b27d1d363 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:17:03.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:17:03.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:17:03.868Z [qtp2045766957-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "3fb43b0c-237f-49b9-8515-4c723378e4df" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "244ea341-a6db-4cce-8507-7871b5ec5d3d.assettoscatemplate" CURR_ARTIFACT_UUID = "244ea341-a6db-4cce-8507-7871b5ec5d3d.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduCp-template.yml,0,244ea341-a6db-4cce-8507-7871b5ec5d3d.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1e51b55c-4013-446c-937c-abcd799aa41f TargetServiceName=null EntryTimestamp=2021-08-11T16:16:51.367Z InvocationID=6d66ba2b-a44a-407d-a8f7-0aa6814b5e8e ErrorCode=500 Starting to create Csar for component VDU Cp 2021-08-11T16:17:04.044Z [qtp2045766957-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "3fb43b0c-237f-49b9-8515-4c723378e4df" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "244ea341-a6db-4cce-8507-7871b5ec5d3d.assettoscatemplate" CURR_ARTIFACT_UUID = "244ea341-a6db-4cce-8507-7871b5ec5d3d.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduCp-template.yml,0,244ea341-a6db-4cce-8507-7871b5ec5d3d.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1e51b55c-4013-446c-937c-abcd799aa41f TargetServiceName=null EntryTimestamp=2021-08-11T16:16:51.367Z InvocationID=6d66ba2b-a44a-407d-a8f7-0aa6814b5e8e ErrorCode=500 Ended create Csar for component VDU Cp 2021-08-11T16:17:04.044Z [qtp2045766957-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "3fb43b0c-237f-49b9-8515-4c723378e4df" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "244ea341-a6db-4cce-8507-7871b5ec5d3d.assettoscatemplate" CURR_ARTIFACT_UUID = "244ea341-a6db-4cce-8507-7871b5ec5d3d.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduCp-template.yml,0,244ea341-a6db-4cce-8507-7871b5ec5d3d.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=1e51b55c-4013-446c-937c-abcd799aa41f TargetServiceName=null EntryTimestamp=2021-08-11T16:16:51.367Z InvocationID=6d66ba2b-a44a-407d-a8f7-0aa6814b5e8e ErrorCode=500 lock resource with id 244ea341-a6db-4cce-8507-7871b5ec5d3d 2021-08-11T16:17:04.265Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:17:05.576Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=434da7f2-7894-46a1-9bd3-410133aff622 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:17:05.576Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=434da7f2-7894-46a1-9bd3-410133aff622 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@9b83609 2021-08-11T16:17:05.576Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=434da7f2-7894-46a1-9bd3-410133aff622 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:17:05.576Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=434da7f2-7894-46a1-9bd3-410133aff622 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:17:06.189Z [qtp2045766957-20] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "3fb43b0c-237f-49b9-8515-4c723378e4df" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "244ea341-a6db-4cce-8507-7871b5ec5d3d.assettoscacsar" CURR_ARTIFACT_UUID = "244ea341-a6db-4cce-8507-7871b5ec5d3d.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VduCp-csar.csar,0,244ea341-a6db-4cce-8507-7871b5ec5d3d.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=14822 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=a657a401-f2e0-4761-b222-37f36d09b35f TargetServiceName=null EntryTimestamp=2021-08-11T16:16:51.367Z InvocationID=6d66ba2b-a44a-407d-a8f7-0aa6814b5e8e ErrorCode=500 LogTimestamp=2021-08-11T16:17:06.189057Z ResponseDesc=Created Exiting. 2021-08-11T16:17:06.189Z [qtp2045766957-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:17:06.269Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=83e01aa9-2147-4d1a-94c3-ffda1953ed1d InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:17:06.269086Z InvocationID=8b1b8926-7227-4b81-b40c-79578e51ffae LogTimestamp=2021-08-11T16:17:06.269136Z Entering 2021-08-11T16:17:06.269Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=83e01aa9-2147-4d1a-94c3-ffda1953ed1d InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:17:06.269086Z InvocationID=8b1b8926-7227-4b81-b40c-79578e51ffae LogTimestamp=2021-08-11T16:17:06.269136Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:17:08.076Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:06.270Z InvocationID=0f405628-eaec-49b2-b87c-0be84eb6dd03 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:17:08.077Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:06.270Z InvocationID=0f405628-eaec-49b2-b87c-0be84eb6dd03 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:17:08.077Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:06.270Z InvocationID=0f405628-eaec-49b2-b87c-0be84eb6dd03 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:17:08.077Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:06.270Z InvocationID=0f405628-eaec-49b2-b87c-0be84eb6dd03 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:17:08.096Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:06.270Z InvocationID=0f405628-eaec-49b2-b87c-0be84eb6dd03 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Vnf 2021-08-11T16:17:08.370Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=348bc4a8-dc8d-4372-810c-52132c1c68e6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:17:08.370Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=348bc4a8-dc8d-4372-810c-52132c1c68e6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6d09a2a9 2021-08-11T16:17:08.370Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=348bc4a8-dc8d-4372-810c-52132c1c68e6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:17:08.370Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=348bc4a8-dc8d-4372-810c-52132c1c68e6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:17:09.455Z [qtp2045766957-12] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "284e2c37-87ac-4d58-b3bc-675b878cb0ff" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VNF" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=284e2c37-87ac-4d58-b3bc-675b878cb0ff EntryTimestamp=2021-08-11T16:17:06.270Z InvocationID=0f405628-eaec-49b2-b87c-0be84eb6dd03 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:17:09.456Z [qtp2045766957-12] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "284e2c37-87ac-4d58-b3bc-675b878cb0ff" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VNF" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=284e2c37-87ac-4d58-b3bc-675b878cb0ff TargetServiceName=null EntryTimestamp=2021-08-11T16:17:06.270Z InvocationID=0f405628-eaec-49b2-b87c-0be84eb6dd03 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:17:09.690Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "284e2c37-87ac-4d58-b3bc-675b878cb0ff" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VNF" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=284e2c37-87ac-4d58-b3bc-675b878cb0ff TargetServiceName=null EntryTimestamp=2021-08-11T16:17:06.270Z InvocationID=0f405628-eaec-49b2-b87c-0be84eb6dd03 ErrorCode=500 lock resource with id 4e1b14a8-767a-4a5d-893c-96c9807d8423 2021-08-11T16:17:10.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=633470d5-e8f7-4dfb-bfcc-fe39cc8f8646 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:17:10.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=633470d5-e8f7-4dfb-bfcc-fe39cc8f8646 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@77ddd636 2021-08-11T16:17:10.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=633470d5-e8f7-4dfb-bfcc-fe39cc8f8646 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:17:10.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=633470d5-e8f7-4dfb-bfcc-fe39cc8f8646 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:17:11.352Z [qtp2045766957-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "284e2c37-87ac-4d58-b3bc-675b878cb0ff" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "4e1b14a8-767a-4a5d-893c-96c9807d8423.assettoscatemplate" CURR_ARTIFACT_UUID = "4e1b14a8-767a-4a5d-893c-96c9807d8423.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vnf-template.yml,0,4e1b14a8-767a-4a5d-893c-96c9807d8423.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ee2d242e-9664-4f75-808d-f0199cda448f TargetServiceName=null EntryTimestamp=2021-08-11T16:17:06.270Z InvocationID=0f405628-eaec-49b2-b87c-0be84eb6dd03 ErrorCode=500 Starting to create Csar for component VNF 2021-08-11T16:17:11.563Z [qtp2045766957-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "284e2c37-87ac-4d58-b3bc-675b878cb0ff" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "4e1b14a8-767a-4a5d-893c-96c9807d8423.assettoscatemplate" CURR_ARTIFACT_UUID = "4e1b14a8-767a-4a5d-893c-96c9807d8423.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vnf-template.yml,0,4e1b14a8-767a-4a5d-893c-96c9807d8423.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ee2d242e-9664-4f75-808d-f0199cda448f TargetServiceName=null EntryTimestamp=2021-08-11T16:17:06.270Z InvocationID=0f405628-eaec-49b2-b87c-0be84eb6dd03 ErrorCode=500 Ended create Csar for component VNF 2021-08-11T16:17:11.563Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "284e2c37-87ac-4d58-b3bc-675b878cb0ff" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "4e1b14a8-767a-4a5d-893c-96c9807d8423.assettoscatemplate" CURR_ARTIFACT_UUID = "4e1b14a8-767a-4a5d-893c-96c9807d8423.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vnf-template.yml,0,4e1b14a8-767a-4a5d-893c-96c9807d8423.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ee2d242e-9664-4f75-808d-f0199cda448f TargetServiceName=null EntryTimestamp=2021-08-11T16:17:06.270Z InvocationID=0f405628-eaec-49b2-b87c-0be84eb6dd03 ErrorCode=500 lock resource with id 4e1b14a8-767a-4a5d-893c-96c9807d8423 2021-08-11T16:17:13.203Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "284e2c37-87ac-4d58-b3bc-675b878cb0ff" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "4e1b14a8-767a-4a5d-893c-96c9807d8423.assettoscacsar" CURR_ARTIFACT_UUID = "4e1b14a8-767a-4a5d-893c-96c9807d8423.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Vnf-csar.csar,0,4e1b14a8-767a-4a5d-893c-96c9807d8423.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=6933 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ba6146c8-bd43-4be4-850a-d2b165155cc1 TargetServiceName=null EntryTimestamp=2021-08-11T16:17:06.270Z InvocationID=0f405628-eaec-49b2-b87c-0be84eb6dd03 ErrorCode=500 LogTimestamp=2021-08-11T16:17:13.203182Z ResponseDesc=Created Exiting. 2021-08-11T16:17:13.204Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:17:13.223Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=6cb2cf7d-1013-43e3-b0df-d1c00d488518 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:17:13.222250Z InvocationID=b6fcd1c7-b345-4f39-975c-cedc9457735f LogTimestamp=2021-08-11T16:17:13.222564Z Entering 2021-08-11T16:17:13.223Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=6cb2cf7d-1013-43e3-b0df-d1c00d488518 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:17:13.222250Z InvocationID=b6fcd1c7-b345-4f39-975c-cedc9457735f LogTimestamp=2021-08-11T16:17:13.222564Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:17:13.683Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:13.223Z InvocationID=9453527c-bee6-4a1d-8595-b0b5838260e6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:17:13.683Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:13.223Z InvocationID=9453527c-bee6-4a1d-8595-b0b5838260e6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:17:13.684Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:13.223Z InvocationID=9453527c-bee6-4a1d-8595-b0b5838260e6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:17:13.684Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:13.223Z InvocationID=9453527c-bee6-4a1d-8595-b0b5838260e6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:17:13.695Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:13.223Z InvocationID=9453527c-bee6-4a1d-8595-b0b5838260e6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name AccessConnectivity 2021-08-11T16:17:13.984Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a92dc7c5-cd9e-4895-8b2b-c2248c4bd6c1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:17:13.984Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a92dc7c5-cd9e-4895-8b2b-c2248c4bd6c1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3bff30dd 2021-08-11T16:17:13.984Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a92dc7c5-cd9e-4895-8b2b-c2248c4bd6c1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:17:13.984Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a92dc7c5-cd9e-4895-8b2b-c2248c4bd6c1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:17:15.173Z [qtp2045766957-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Access Connectivity" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "28f84f10-0422-4bb6-8739-8f2274dc8e19" INVARIANT_UUID = "af1a9bbf-8266-466b-af69-8aa8b2a70fea" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.accessConnectivity" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=28f84f10-0422-4bb6-8739-8f2274dc8e19 EntryTimestamp=2021-08-11T16:17:13.223Z InvocationID=9453527c-bee6-4a1d-8595-b0b5838260e6 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:17:15.173Z [qtp2045766957-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Access Connectivity" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "28f84f10-0422-4bb6-8739-8f2274dc8e19" INVARIANT_UUID = "af1a9bbf-8266-466b-af69-8aa8b2a70fea" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.accessConnectivity" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=28f84f10-0422-4bb6-8739-8f2274dc8e19 TargetServiceName=null EntryTimestamp=2021-08-11T16:17:13.223Z InvocationID=9453527c-bee6-4a1d-8595-b0b5838260e6 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:17:15.693Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Access Connectivity" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "28f84f10-0422-4bb6-8739-8f2274dc8e19" INVARIANT_UUID = "af1a9bbf-8266-466b-af69-8aa8b2a70fea" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.accessConnectivity" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=28f84f10-0422-4bb6-8739-8f2274dc8e19 TargetServiceName=null EntryTimestamp=2021-08-11T16:17:13.223Z InvocationID=9453527c-bee6-4a1d-8595-b0b5838260e6 ErrorCode=500 lock resource with id 5592a43d-140b-4a05-9249-8c9fb696d7fc 2021-08-11T16:17:16.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44e210d8-17a6-4624-a2b2-9388bde8b2b1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:17:16.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44e210d8-17a6-4624-a2b2-9388bde8b2b1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@73bf51d5 2021-08-11T16:17:16.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44e210d8-17a6-4624-a2b2-9388bde8b2b1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:17:16.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44e210d8-17a6-4624-a2b2-9388bde8b2b1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:17:17.403Z [qtp2045766957-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Access Connectivity" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "28f84f10-0422-4bb6-8739-8f2274dc8e19" INVARIANT_UUID = "af1a9bbf-8266-466b-af69-8aa8b2a70fea" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "5592a43d-140b-4a05-9249-8c9fb696d7fc.assettoscatemplate" CURR_ARTIFACT_UUID = "5592a43d-140b-4a05-9249-8c9fb696d7fc.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-AccessConnectivity-template.yml,0,5592a43d-140b-4a05-9249-8c9fb696d7fc.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=67ceec99-041d-435a-9e76-3ee17e539528 TargetServiceName=null EntryTimestamp=2021-08-11T16:17:13.223Z InvocationID=9453527c-bee6-4a1d-8595-b0b5838260e6 ErrorCode=500 Starting to create Csar for component Access Connectivity 2021-08-11T16:17:17.616Z [qtp2045766957-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Access Connectivity" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "28f84f10-0422-4bb6-8739-8f2274dc8e19" INVARIANT_UUID = "af1a9bbf-8266-466b-af69-8aa8b2a70fea" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "5592a43d-140b-4a05-9249-8c9fb696d7fc.assettoscatemplate" CURR_ARTIFACT_UUID = "5592a43d-140b-4a05-9249-8c9fb696d7fc.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-AccessConnectivity-template.yml,0,5592a43d-140b-4a05-9249-8c9fb696d7fc.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=67ceec99-041d-435a-9e76-3ee17e539528 TargetServiceName=null EntryTimestamp=2021-08-11T16:17:13.223Z InvocationID=9453527c-bee6-4a1d-8595-b0b5838260e6 ErrorCode=500 Ended create Csar for component Access Connectivity 2021-08-11T16:17:17.616Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Access Connectivity" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "28f84f10-0422-4bb6-8739-8f2274dc8e19" INVARIANT_UUID = "af1a9bbf-8266-466b-af69-8aa8b2a70fea" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "5592a43d-140b-4a05-9249-8c9fb696d7fc.assettoscatemplate" CURR_ARTIFACT_UUID = "5592a43d-140b-4a05-9249-8c9fb696d7fc.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-AccessConnectivity-template.yml,0,5592a43d-140b-4a05-9249-8c9fb696d7fc.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=67ceec99-041d-435a-9e76-3ee17e539528 TargetServiceName=null EntryTimestamp=2021-08-11T16:17:13.223Z InvocationID=9453527c-bee6-4a1d-8595-b0b5838260e6 ErrorCode=500 lock resource with id 5592a43d-140b-4a05-9249-8c9fb696d7fc 2021-08-11T16:17:18.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:17:18.316Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:17:19.390Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Access Connectivity" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "28f84f10-0422-4bb6-8739-8f2274dc8e19" INVARIANT_UUID = "af1a9bbf-8266-466b-af69-8aa8b2a70fea" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "5592a43d-140b-4a05-9249-8c9fb696d7fc.assettoscacsar" CURR_ARTIFACT_UUID = "5592a43d-140b-4a05-9249-8c9fb696d7fc.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-AccessConnectivity-csar.csar,0,5592a43d-140b-4a05-9249-8c9fb696d7fc.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=6166 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=72b7cbbe-31f6-4963-b545-4ef1ce10554d TargetServiceName=null EntryTimestamp=2021-08-11T16:17:13.223Z InvocationID=9453527c-bee6-4a1d-8595-b0b5838260e6 ErrorCode=500 LogTimestamp=2021-08-11T16:17:19.389984Z ResponseDesc=Created Exiting. 2021-08-11T16:17:19.391Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:17:19.453Z [qtp2045766957-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=e497cc0c-46bd-4412-a79f-3e25a2f2d829 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:17:19.453217Z InvocationID=2a9352fa-ae17-45e7-bbac-fbadebd63189 LogTimestamp=2021-08-11T16:17:19.453290Z Entering 2021-08-11T16:17:19.453Z [qtp2045766957-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=e497cc0c-46bd-4412-a79f-3e25a2f2d829 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:17:19.453217Z InvocationID=2a9352fa-ae17-45e7-bbac-fbadebd63189 LogTimestamp=2021-08-11T16:17:19.453290Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:17:19.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d9d8ebd-bfd6-4b25-84d2-c635d2adbccf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:17:19.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d9d8ebd-bfd6-4b25-84d2-c635d2adbccf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5383b9e1 2021-08-11T16:17:19.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d9d8ebd-bfd6-4b25-84d2-c635d2adbccf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:17:19.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d9d8ebd-bfd6-4b25-84d2-c635d2adbccf PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:17:20.277Z [qtp2045766957-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:19.454Z InvocationID=f14e3230-9d52-471a-819c-1039757de8a1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:17:20.277Z [qtp2045766957-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:19.454Z InvocationID=f14e3230-9d52-471a-819c-1039757de8a1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:17:20.277Z [qtp2045766957-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:19.454Z InvocationID=f14e3230-9d52-471a-819c-1039757de8a1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:17:20.277Z [qtp2045766957-20] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:19.454Z InvocationID=f14e3230-9d52-471a-819c-1039757de8a1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:17:20.289Z [qtp2045766957-20] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:19.454Z InvocationID=f14e3230-9d52-471a-819c-1039757de8a1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Ontpnf 2021-08-11T16:17:21.502Z [qtp2045766957-20] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "OntPnf" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "2543a875-2dbe-4e72-a66f-cfe26e383693" INVARIANT_UUID = "ecdf26a4-81e0-4004-b7b9-942aeea9a517" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.OntPnf" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2543a875-2dbe-4e72-a66f-cfe26e383693 EntryTimestamp=2021-08-11T16:17:19.454Z InvocationID=f14e3230-9d52-471a-819c-1039757de8a1 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:17:21.502Z [qtp2045766957-20] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "OntPnf" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "2543a875-2dbe-4e72-a66f-cfe26e383693" INVARIANT_UUID = "ecdf26a4-81e0-4004-b7b9-942aeea9a517" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.OntPnf" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2543a875-2dbe-4e72-a66f-cfe26e383693 TargetServiceName=null EntryTimestamp=2021-08-11T16:17:19.454Z InvocationID=f14e3230-9d52-471a-819c-1039757de8a1 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:17:21.801Z [qtp2045766957-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "OntPnf" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "2543a875-2dbe-4e72-a66f-cfe26e383693" INVARIANT_UUID = "ecdf26a4-81e0-4004-b7b9-942aeea9a517" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.vfc.OntPnf" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2543a875-2dbe-4e72-a66f-cfe26e383693 TargetServiceName=null EntryTimestamp=2021-08-11T16:17:19.454Z InvocationID=f14e3230-9d52-471a-819c-1039757de8a1 ErrorCode=500 lock resource with id f34a4466-93b1-45bc-8bc0-bb7282ce8a1a 2021-08-11T16:17:22.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1980c23b-dd75-43cf-8809-cc57f30b8531 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:17:22.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1980c23b-dd75-43cf-8809-cc57f30b8531 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4a76a9c1 2021-08-11T16:17:22.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1980c23b-dd75-43cf-8809-cc57f30b8531 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:17:22.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1980c23b-dd75-43cf-8809-cc57f30b8531 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:17:23.412Z [qtp2045766957-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "OntPnf" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "2543a875-2dbe-4e72-a66f-cfe26e383693" INVARIANT_UUID = "ecdf26a4-81e0-4004-b7b9-942aeea9a517" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "f34a4466-93b1-45bc-8bc0-bb7282ce8a1a.assettoscatemplate" CURR_ARTIFACT_UUID = "f34a4466-93b1-45bc-8bc0-bb7282ce8a1a.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Ontpnf-template.yml,0,f34a4466-93b1-45bc-8bc0-bb7282ce8a1a.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=27f9be01-24b6-4e2e-9396-fc86b914e4e6 TargetServiceName=null EntryTimestamp=2021-08-11T16:17:19.454Z InvocationID=f14e3230-9d52-471a-819c-1039757de8a1 ErrorCode=500 Starting to create Csar for component OntPnf 2021-08-11T16:17:23.630Z [qtp2045766957-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "OntPnf" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "2543a875-2dbe-4e72-a66f-cfe26e383693" INVARIANT_UUID = "ecdf26a4-81e0-4004-b7b9-942aeea9a517" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "f34a4466-93b1-45bc-8bc0-bb7282ce8a1a.assettoscatemplate" CURR_ARTIFACT_UUID = "f34a4466-93b1-45bc-8bc0-bb7282ce8a1a.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Ontpnf-template.yml,0,f34a4466-93b1-45bc-8bc0-bb7282ce8a1a.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=27f9be01-24b6-4e2e-9396-fc86b914e4e6 TargetServiceName=null EntryTimestamp=2021-08-11T16:17:19.454Z InvocationID=f14e3230-9d52-471a-819c-1039757de8a1 ErrorCode=500 Ended create Csar for component OntPnf 2021-08-11T16:17:23.630Z [qtp2045766957-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "OntPnf" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "2543a875-2dbe-4e72-a66f-cfe26e383693" INVARIANT_UUID = "ecdf26a4-81e0-4004-b7b9-942aeea9a517" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "f34a4466-93b1-45bc-8bc0-bb7282ce8a1a.assettoscatemplate" CURR_ARTIFACT_UUID = "f34a4466-93b1-45bc-8bc0-bb7282ce8a1a.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Ontpnf-template.yml,0,f34a4466-93b1-45bc-8bc0-bb7282ce8a1a.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=27f9be01-24b6-4e2e-9396-fc86b914e4e6 TargetServiceName=null EntryTimestamp=2021-08-11T16:17:19.454Z InvocationID=f14e3230-9d52-471a-819c-1039757de8a1 ErrorCode=500 lock resource with id f34a4466-93b1-45bc-8bc0-bb7282ce8a1a 2021-08-11T16:17:25.274Z [qtp2045766957-20] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "OntPnf" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "2543a875-2dbe-4e72-a66f-cfe26e383693" INVARIANT_UUID = "ecdf26a4-81e0-4004-b7b9-942aeea9a517" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "f34a4466-93b1-45bc-8bc0-bb7282ce8a1a.assettoscacsar" CURR_ARTIFACT_UUID = "f34a4466-93b1-45bc-8bc0-bb7282ce8a1a.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Ontpnf-csar.csar,0,f34a4466-93b1-45bc-8bc0-bb7282ce8a1a.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5820 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f8afe515-dfbf-4200-a7a9-7dcc0fbfa138 TargetServiceName=null EntryTimestamp=2021-08-11T16:17:19.454Z InvocationID=f14e3230-9d52-471a-819c-1039757de8a1 ErrorCode=500 LogTimestamp=2021-08-11T16:17:25.274405Z ResponseDesc=Created Exiting. 2021-08-11T16:17:25.275Z [qtp2045766957-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:17:25.290Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=d63f5449-f2d9-418a-b6c0-d3cd5f825ce4 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:17:25.290541Z InvocationID=1ff8afc6-56b9-442f-b65a-d16656e23de5 LogTimestamp=2021-08-11T16:17:25.290580Z Entering 2021-08-11T16:17:25.290Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=d63f5449-f2d9-418a-b6c0-d3cd5f825ce4 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:17:25.290541Z InvocationID=1ff8afc6-56b9-442f-b65a-d16656e23de5 LogTimestamp=2021-08-11T16:17:25.290580Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:17:25.751Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ef63044-b203-4485-9b6b-1b1f99367593 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:17:25.751Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ef63044-b203-4485-9b6b-1b1f99367593 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@84d018a 2021-08-11T16:17:25.751Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ef63044-b203-4485-9b6b-1b1f99367593 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:17:25.751Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ef63044-b203-4485-9b6b-1b1f99367593 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:17:25.998Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:25.291Z InvocationID=1a7d17f3-2721-47ef-92be-2fd9b673c2a2 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:17:25.998Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:25.291Z InvocationID=1a7d17f3-2721-47ef-92be-2fd9b673c2a2 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:17:25.998Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:25.291Z InvocationID=1a7d17f3-2721-47ef-92be-2fd9b673c2a2 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:17:25.998Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:25.291Z InvocationID=1a7d17f3-2721-47ef-92be-2fd9b673c2a2 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:17:26.075Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:25.291Z InvocationID=1a7d17f3-2721-47ef-92be-2fd9b673c2a2 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Ponuni 2021-08-11T16:17:27.956Z [qtp2045766957-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "PonUni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "0eb94c4f-42d3-4bd0-ba4a-289f106ab7ca" INVARIANT_UUID = "a3ad718e-69fe-437e-a747-79de64ed372e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.PonUni" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=0eb94c4f-42d3-4bd0-ba4a-289f106ab7ca EntryTimestamp=2021-08-11T16:17:25.291Z InvocationID=1a7d17f3-2721-47ef-92be-2fd9b673c2a2 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:17:27.956Z [qtp2045766957-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "PonUni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "0eb94c4f-42d3-4bd0-ba4a-289f106ab7ca" INVARIANT_UUID = "a3ad718e-69fe-437e-a747-79de64ed372e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.PonUni" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=0eb94c4f-42d3-4bd0-ba4a-289f106ab7ca TargetServiceName=null EntryTimestamp=2021-08-11T16:17:25.291Z InvocationID=1a7d17f3-2721-47ef-92be-2fd9b673c2a2 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:17:28.667Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "PonUni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "0eb94c4f-42d3-4bd0-ba4a-289f106ab7ca" INVARIANT_UUID = "a3ad718e-69fe-437e-a747-79de64ed372e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.PonUni" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=0eb94c4f-42d3-4bd0-ba4a-289f106ab7ca TargetServiceName=null EntryTimestamp=2021-08-11T16:17:25.291Z InvocationID=1a7d17f3-2721-47ef-92be-2fd9b673c2a2 ErrorCode=500 lock resource with id c070c368-8fd5-460e-9e54-221ad89905a8 2021-08-11T16:17:28.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f39ad46a-7233-47e7-8570-30dcec094515 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:17:28.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f39ad46a-7233-47e7-8570-30dcec094515 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@175c8af 2021-08-11T16:17:28.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f39ad46a-7233-47e7-8570-30dcec094515 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:17:28.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f39ad46a-7233-47e7-8570-30dcec094515 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:17:30.552Z [qtp2045766957-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "PonUni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "0eb94c4f-42d3-4bd0-ba4a-289f106ab7ca" INVARIANT_UUID = "a3ad718e-69fe-437e-a747-79de64ed372e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c070c368-8fd5-460e-9e54-221ad89905a8.assettoscatemplate" CURR_ARTIFACT_UUID = "c070c368-8fd5-460e-9e54-221ad89905a8.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Ponuni-template.yml,0,c070c368-8fd5-460e-9e54-221ad89905a8.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=34a76e6b-488d-4227-9202-91cd691612ee TargetServiceName=null EntryTimestamp=2021-08-11T16:17:25.291Z InvocationID=1a7d17f3-2721-47ef-92be-2fd9b673c2a2 ErrorCode=500 Starting to create Csar for component PonUni 2021-08-11T16:17:30.744Z [qtp2045766957-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "PonUni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "0eb94c4f-42d3-4bd0-ba4a-289f106ab7ca" INVARIANT_UUID = "a3ad718e-69fe-437e-a747-79de64ed372e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c070c368-8fd5-460e-9e54-221ad89905a8.assettoscatemplate" CURR_ARTIFACT_UUID = "c070c368-8fd5-460e-9e54-221ad89905a8.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Ponuni-template.yml,0,c070c368-8fd5-460e-9e54-221ad89905a8.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=34a76e6b-488d-4227-9202-91cd691612ee TargetServiceName=null EntryTimestamp=2021-08-11T16:17:25.291Z InvocationID=1a7d17f3-2721-47ef-92be-2fd9b673c2a2 ErrorCode=500 Ended create Csar for component PonUni 2021-08-11T16:17:30.744Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "PonUni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "0eb94c4f-42d3-4bd0-ba4a-289f106ab7ca" INVARIANT_UUID = "a3ad718e-69fe-437e-a747-79de64ed372e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c070c368-8fd5-460e-9e54-221ad89905a8.assettoscatemplate" CURR_ARTIFACT_UUID = "c070c368-8fd5-460e-9e54-221ad89905a8.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Ponuni-template.yml,0,c070c368-8fd5-460e-9e54-221ad89905a8.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=34a76e6b-488d-4227-9202-91cd691612ee TargetServiceName=null EntryTimestamp=2021-08-11T16:17:25.291Z InvocationID=1a7d17f3-2721-47ef-92be-2fd9b673c2a2 ErrorCode=500 lock resource with id c070c368-8fd5-460e-9e54-221ad89905a8 2021-08-11T16:17:31.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a7696999-72de-4d99-ba69-156f5401a9e3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:17:31.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a7696999-72de-4d99-ba69-156f5401a9e3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4ff735d1 2021-08-11T16:17:31.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a7696999-72de-4d99-ba69-156f5401a9e3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:17:31.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a7696999-72de-4d99-ba69-156f5401a9e3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:17:32.434Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "PonUni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "0eb94c4f-42d3-4bd0-ba4a-289f106ab7ca" INVARIANT_UUID = "a3ad718e-69fe-437e-a747-79de64ed372e" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "c070c368-8fd5-460e-9e54-221ad89905a8.assettoscacsar" CURR_ARTIFACT_UUID = "c070c368-8fd5-460e-9e54-221ad89905a8.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Ponuni-csar.csar,0,c070c368-8fd5-460e-9e54-221ad89905a8.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=7143 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b3fec83a-4135-4b20-9f06-ea67a6515ff9 TargetServiceName=null EntryTimestamp=2021-08-11T16:17:25.291Z InvocationID=1a7d17f3-2721-47ef-92be-2fd9b673c2a2 ErrorCode=500 LogTimestamp=2021-08-11T16:17:32.434145Z ResponseDesc=Created Exiting. 2021-08-11T16:17:32.435Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:17:32.457Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=8e17822a-429c-493e-8981-889610ef8fc4 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:17:32.456934Z InvocationID=0723f232-2537-46d2-9497-946cddf52007 LogTimestamp=2021-08-11T16:17:32.457001Z Entering 2021-08-11T16:17:32.457Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=8e17822a-429c-493e-8981-889610ef8fc4 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:17:32.456934Z InvocationID=0723f232-2537-46d2-9497-946cddf52007 LogTimestamp=2021-08-11T16:17:32.457001Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:17:32.768Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:32.458Z InvocationID=f7c42cda-067b-4a02-b857-2cae968036c8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:17:32.768Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:32.458Z InvocationID=f7c42cda-067b-4a02-b857-2cae968036c8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:17:32.768Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:32.458Z InvocationID=f7c42cda-067b-4a02-b857-2cae968036c8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:17:32.768Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:32.458Z InvocationID=f7c42cda-067b-4a02-b857-2cae968036c8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:17:32.804Z [qtp2045766957-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:32.458Z InvocationID=f7c42cda-067b-4a02-b857-2cae968036c8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Oltnni 2021-08-11T16:17:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:17:33.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:17:34.073Z [qtp2045766957-15] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "OltNni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "2ba81436-72b7-4ea4-bf54-d7fe668c2186" INVARIANT_UUID = "676644e6-b637-4e42-a00b-45ba737b3215" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.OltNni" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2ba81436-72b7-4ea4-bf54-d7fe668c2186 EntryTimestamp=2021-08-11T16:17:32.458Z InvocationID=f7c42cda-067b-4a02-b857-2cae968036c8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:17:34.073Z [qtp2045766957-15] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "OltNni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "2ba81436-72b7-4ea4-bf54-d7fe668c2186" INVARIANT_UUID = "676644e6-b637-4e42-a00b-45ba737b3215" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.OltNni" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2ba81436-72b7-4ea4-bf54-d7fe668c2186 TargetServiceName=null EntryTimestamp=2021-08-11T16:17:32.458Z InvocationID=f7c42cda-067b-4a02-b857-2cae968036c8 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:17:34.385Z [qtp2045766957-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "OltNni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "2ba81436-72b7-4ea4-bf54-d7fe668c2186" INVARIANT_UUID = "676644e6-b637-4e42-a00b-45ba737b3215" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.OltNni" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2ba81436-72b7-4ea4-bf54-d7fe668c2186 TargetServiceName=null EntryTimestamp=2021-08-11T16:17:32.458Z InvocationID=f7c42cda-067b-4a02-b857-2cae968036c8 ErrorCode=500 lock resource with id d0a36627-1356-4e0c-bd31-2a0637ce733d 2021-08-11T16:17:34.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a392d34c-6c8b-491f-9b5a-f7fbc3d09bbf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:17:34.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a392d34c-6c8b-491f-9b5a-f7fbc3d09bbf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3030b54f 2021-08-11T16:17:34.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a392d34c-6c8b-491f-9b5a-f7fbc3d09bbf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:17:34.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a392d34c-6c8b-491f-9b5a-f7fbc3d09bbf PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:17:35.998Z [qtp2045766957-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "OltNni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "2ba81436-72b7-4ea4-bf54-d7fe668c2186" INVARIANT_UUID = "676644e6-b637-4e42-a00b-45ba737b3215" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "d0a36627-1356-4e0c-bd31-2a0637ce733d.assettoscatemplate" CURR_ARTIFACT_UUID = "d0a36627-1356-4e0c-bd31-2a0637ce733d.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Oltnni-template.yml,0,d0a36627-1356-4e0c-bd31-2a0637ce733d.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=292823b2-92b0-4190-8811-4fc0be6176e4 TargetServiceName=null EntryTimestamp=2021-08-11T16:17:32.458Z InvocationID=f7c42cda-067b-4a02-b857-2cae968036c8 ErrorCode=500 Starting to create Csar for component OltNni 2021-08-11T16:17:36.178Z [qtp2045766957-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "OltNni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "2ba81436-72b7-4ea4-bf54-d7fe668c2186" INVARIANT_UUID = "676644e6-b637-4e42-a00b-45ba737b3215" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "d0a36627-1356-4e0c-bd31-2a0637ce733d.assettoscatemplate" CURR_ARTIFACT_UUID = "d0a36627-1356-4e0c-bd31-2a0637ce733d.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Oltnni-template.yml,0,d0a36627-1356-4e0c-bd31-2a0637ce733d.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=292823b2-92b0-4190-8811-4fc0be6176e4 TargetServiceName=null EntryTimestamp=2021-08-11T16:17:32.458Z InvocationID=f7c42cda-067b-4a02-b857-2cae968036c8 ErrorCode=500 Ended create Csar for component OltNni 2021-08-11T16:17:36.179Z [qtp2045766957-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "OltNni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "2ba81436-72b7-4ea4-bf54-d7fe668c2186" INVARIANT_UUID = "676644e6-b637-4e42-a00b-45ba737b3215" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "d0a36627-1356-4e0c-bd31-2a0637ce733d.assettoscatemplate" CURR_ARTIFACT_UUID = "d0a36627-1356-4e0c-bd31-2a0637ce733d.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Oltnni-template.yml,0,d0a36627-1356-4e0c-bd31-2a0637ce733d.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=292823b2-92b0-4190-8811-4fc0be6176e4 TargetServiceName=null EntryTimestamp=2021-08-11T16:17:32.458Z InvocationID=f7c42cda-067b-4a02-b857-2cae968036c8 ErrorCode=500 lock resource with id d0a36627-1356-4e0c-bd31-2a0637ce733d 2021-08-11T16:17:37.688Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f9bc134-66a7-471c-b558-a0c4015884c2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:17:37.688Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f9bc134-66a7-471c-b558-a0c4015884c2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5011a392 2021-08-11T16:17:37.688Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f9bc134-66a7-471c-b558-a0c4015884c2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:17:37.688Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f9bc134-66a7-471c-b558-a0c4015884c2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:17:37.856Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "OltNni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "2ba81436-72b7-4ea4-bf54-d7fe668c2186" INVARIANT_UUID = "676644e6-b637-4e42-a00b-45ba737b3215" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "d0a36627-1356-4e0c-bd31-2a0637ce733d.assettoscacsar" CURR_ARTIFACT_UUID = "d0a36627-1356-4e0c-bd31-2a0637ce733d.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Oltnni-csar.csar,0,d0a36627-1356-4e0c-bd31-2a0637ce733d.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5398 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=81d8725c-5249-4809-a0ed-a584db248750 TargetServiceName=null EntryTimestamp=2021-08-11T16:17:32.458Z InvocationID=f7c42cda-067b-4a02-b857-2cae968036c8 ErrorCode=500 LogTimestamp=2021-08-11T16:17:37.856422Z ResponseDesc=Created Exiting. 2021-08-11T16:17:37.857Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:17:37.877Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=acb74a87-d00e-4f0f-b463-86a70df54a67 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:17:37.876735Z InvocationID=6dc87876-b303-4f33-ade6-ac7bc8915141 LogTimestamp=2021-08-11T16:17:37.876802Z Entering 2021-08-11T16:17:37.877Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=acb74a87-d00e-4f0f-b463-86a70df54a67 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:17:37.876735Z InvocationID=6dc87876-b303-4f33-ade6-ac7bc8915141 LogTimestamp=2021-08-11T16:17:37.876802Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:17:39.073Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:37.877Z InvocationID=0ac1b447-cac0-49ad-8945-a1b1591e23a8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:17:39.073Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:37.877Z InvocationID=0ac1b447-cac0-49ad-8945-a1b1591e23a8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:17:39.073Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:37.877Z InvocationID=0ac1b447-cac0-49ad-8945-a1b1591e23a8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:17:39.073Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:37.877Z InvocationID=0ac1b447-cac0-49ad-8945-a1b1591e23a8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:17:39.160Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:37.877Z InvocationID=0ac1b447-cac0-49ad-8945-a1b1591e23a8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Ontnni 2021-08-11T16:17:40.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94278fdc-54ce-451f-ae2b-60e32317658a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:17:40.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94278fdc-54ce-451f-ae2b-60e32317658a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7f8e298f 2021-08-11T16:17:40.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94278fdc-54ce-451f-ae2b-60e32317658a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:17:40.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94278fdc-54ce-451f-ae2b-60e32317658a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:17:41.267Z [qtp2045766957-14] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "OntNni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "72185f0b-f115-48a7-935c-20a217aba228" INVARIANT_UUID = "7839c42b-73fe-4dcf-abea-b8732be78f5a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.OntNni" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=72185f0b-f115-48a7-935c-20a217aba228 EntryTimestamp=2021-08-11T16:17:37.877Z InvocationID=0ac1b447-cac0-49ad-8945-a1b1591e23a8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:17:41.267Z [qtp2045766957-14] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "OntNni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "72185f0b-f115-48a7-935c-20a217aba228" INVARIANT_UUID = "7839c42b-73fe-4dcf-abea-b8732be78f5a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.OntNni" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=72185f0b-f115-48a7-935c-20a217aba228 TargetServiceName=null EntryTimestamp=2021-08-11T16:17:37.877Z InvocationID=0ac1b447-cac0-49ad-8945-a1b1591e23a8 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:17:42.393Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "OntNni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "72185f0b-f115-48a7-935c-20a217aba228" INVARIANT_UUID = "7839c42b-73fe-4dcf-abea-b8732be78f5a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "org.openecomp.resource.cp.OntNni" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=72185f0b-f115-48a7-935c-20a217aba228 TargetServiceName=null EntryTimestamp=2021-08-11T16:17:37.877Z InvocationID=0ac1b447-cac0-49ad-8945-a1b1591e23a8 ErrorCode=500 lock resource with id 19a4717d-3a52-4a36-9f6e-b557060513dd 2021-08-11T16:17:43.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fbea3291-5d2b-4643-9dc9-25269a0b94b9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:17:43.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fbea3291-5d2b-4643-9dc9-25269a0b94b9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@9a1d212 2021-08-11T16:17:43.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fbea3291-5d2b-4643-9dc9-25269a0b94b9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:17:43.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fbea3291-5d2b-4643-9dc9-25269a0b94b9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:17:44.005Z [qtp2045766957-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "OntNni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "72185f0b-f115-48a7-935c-20a217aba228" INVARIANT_UUID = "7839c42b-73fe-4dcf-abea-b8732be78f5a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "19a4717d-3a52-4a36-9f6e-b557060513dd.assettoscatemplate" CURR_ARTIFACT_UUID = "19a4717d-3a52-4a36-9f6e-b557060513dd.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Ontnni-template.yml,0,19a4717d-3a52-4a36-9f6e-b557060513dd.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e8625126-44de-4f22-bc09-1790cf07bb89 TargetServiceName=null EntryTimestamp=2021-08-11T16:17:37.877Z InvocationID=0ac1b447-cac0-49ad-8945-a1b1591e23a8 ErrorCode=500 Starting to create Csar for component OntNni 2021-08-11T16:17:44.211Z [qtp2045766957-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "OntNni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "72185f0b-f115-48a7-935c-20a217aba228" INVARIANT_UUID = "7839c42b-73fe-4dcf-abea-b8732be78f5a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "19a4717d-3a52-4a36-9f6e-b557060513dd.assettoscatemplate" CURR_ARTIFACT_UUID = "19a4717d-3a52-4a36-9f6e-b557060513dd.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Ontnni-template.yml,0,19a4717d-3a52-4a36-9f6e-b557060513dd.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e8625126-44de-4f22-bc09-1790cf07bb89 TargetServiceName=null EntryTimestamp=2021-08-11T16:17:37.877Z InvocationID=0ac1b447-cac0-49ad-8945-a1b1591e23a8 ErrorCode=500 Ended create Csar for component OntNni 2021-08-11T16:17:44.211Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "OntNni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "72185f0b-f115-48a7-935c-20a217aba228" INVARIANT_UUID = "7839c42b-73fe-4dcf-abea-b8732be78f5a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "19a4717d-3a52-4a36-9f6e-b557060513dd.assettoscatemplate" CURR_ARTIFACT_UUID = "19a4717d-3a52-4a36-9f6e-b557060513dd.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Ontnni-template.yml,0,19a4717d-3a52-4a36-9f6e-b557060513dd.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e8625126-44de-4f22-bc09-1790cf07bb89 TargetServiceName=null EntryTimestamp=2021-08-11T16:17:37.877Z InvocationID=0ac1b447-cac0-49ad-8945-a1b1591e23a8 ErrorCode=500 lock resource with id 19a4717d-3a52-4a36-9f6e-b557060513dd 2021-08-11T16:17:45.867Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "OntNni" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "72185f0b-f115-48a7-935c-20a217aba228" INVARIANT_UUID = "7839c42b-73fe-4dcf-abea-b8732be78f5a" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "19a4717d-3a52-4a36-9f6e-b557060513dd.assettoscacsar" CURR_ARTIFACT_UUID = "19a4717d-3a52-4a36-9f6e-b557060513dd.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Ontnni-csar.csar,0,19a4717d-3a52-4a36-9f6e-b557060513dd.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=7990 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=28f63e37-85d5-455a-9ed0-eed84070f1fd TargetServiceName=null EntryTimestamp=2021-08-11T16:17:37.877Z InvocationID=0ac1b447-cac0-49ad-8945-a1b1591e23a8 ErrorCode=500 LogTimestamp=2021-08-11T16:17:45.867028Z ResponseDesc=Created Exiting. 2021-08-11T16:17:45.868Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:17:45.888Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=3f2aeb37-c060-4db5-883e-81589744af95 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:17:45.888702Z InvocationID=ce026248-a19a-4475-a6e3-c7ef8aecad76 LogTimestamp=2021-08-11T16:17:45.888755Z Entering 2021-08-11T16:17:45.889Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=3f2aeb37-c060-4db5-883e-81589744af95 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:17:45.888702Z InvocationID=ce026248-a19a-4475-a6e3-c7ef8aecad76 LogTimestamp=2021-08-11T16:17:45.888755Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:17:46.355Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:45.889Z InvocationID=b0e4afaa-fd53-4980-9374-fda3af9cfd1d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:17:46.355Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:45.889Z InvocationID=b0e4afaa-fd53-4980-9374-fda3af9cfd1d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:17:46.355Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:45.889Z InvocationID=b0e4afaa-fd53-4980-9374-fda3af9cfd1d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:17:46.355Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:45.889Z InvocationID=b0e4afaa-fd53-4980-9374-fda3af9cfd1d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:17:46.366Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:45.889Z InvocationID=b0e4afaa-fd53-4980-9374-fda3af9cfd1d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Sap 2021-08-11T16:17:46.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d352025e-9338-476a-9ebc-32f16784e4fc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:17:46.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d352025e-9338-476a-9ebc-32f16784e4fc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3c8e94a4 2021-08-11T16:17:46.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d352025e-9338-476a-9ebc-32f16784e4fc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:17:46.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d352025e-9338-476a-9ebc-32f16784e4fc PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:17:47.773Z [qtp2045766957-12] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Sap" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "9a01a61b-4650-491b-8d3a-3f65c8aba7e8" INVARIANT_UUID = "2e85475e-9148-4a21-806e-aae059385867" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Sap" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9a01a61b-4650-491b-8d3a-3f65c8aba7e8 EntryTimestamp=2021-08-11T16:17:45.889Z InvocationID=b0e4afaa-fd53-4980-9374-fda3af9cfd1d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:17:47.774Z [qtp2045766957-12] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Sap" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "9a01a61b-4650-491b-8d3a-3f65c8aba7e8" INVARIANT_UUID = "2e85475e-9148-4a21-806e-aae059385867" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Sap" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9a01a61b-4650-491b-8d3a-3f65c8aba7e8 TargetServiceName=null EntryTimestamp=2021-08-11T16:17:45.889Z InvocationID=b0e4afaa-fd53-4980-9374-fda3af9cfd1d ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:17:48.297Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Sap" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "9a01a61b-4650-491b-8d3a-3f65c8aba7e8" INVARIANT_UUID = "2e85475e-9148-4a21-806e-aae059385867" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Sap" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9a01a61b-4650-491b-8d3a-3f65c8aba7e8 TargetServiceName=null EntryTimestamp=2021-08-11T16:17:45.889Z InvocationID=b0e4afaa-fd53-4980-9374-fda3af9cfd1d ErrorCode=500 lock resource with id ff58d4ec-6a95-4413-bc95-e123bc65cfb8 2021-08-11T16:17:48.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:17:48.321Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:17:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:17:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:17:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:17:50.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=032f22e3-d591-46fd-8147-415cebb3198a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:17:50.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=032f22e3-d591-46fd-8147-415cebb3198a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@615ead94 2021-08-11T16:17:50.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=032f22e3-d591-46fd-8147-415cebb3198a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:17:50.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=032f22e3-d591-46fd-8147-415cebb3198a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:17:50.367Z [qtp2045766957-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Sap" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "9a01a61b-4650-491b-8d3a-3f65c8aba7e8" INVARIANT_UUID = "2e85475e-9148-4a21-806e-aae059385867" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "ff58d4ec-6a95-4413-bc95-e123bc65cfb8.assettoscatemplate" CURR_ARTIFACT_UUID = "ff58d4ec-6a95-4413-bc95-e123bc65cfb8.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Sap-template.yml,0,ff58d4ec-6a95-4413-bc95-e123bc65cfb8.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d8a95055-e2cd-4fa9-b33e-c8fe370b3610 TargetServiceName=null EntryTimestamp=2021-08-11T16:17:45.889Z InvocationID=b0e4afaa-fd53-4980-9374-fda3af9cfd1d ErrorCode=500 Starting to create Csar for component Sap 2021-08-11T16:17:50.535Z [qtp2045766957-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Sap" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "9a01a61b-4650-491b-8d3a-3f65c8aba7e8" INVARIANT_UUID = "2e85475e-9148-4a21-806e-aae059385867" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "ff58d4ec-6a95-4413-bc95-e123bc65cfb8.assettoscatemplate" CURR_ARTIFACT_UUID = "ff58d4ec-6a95-4413-bc95-e123bc65cfb8.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Sap-template.yml,0,ff58d4ec-6a95-4413-bc95-e123bc65cfb8.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d8a95055-e2cd-4fa9-b33e-c8fe370b3610 TargetServiceName=null EntryTimestamp=2021-08-11T16:17:45.889Z InvocationID=b0e4afaa-fd53-4980-9374-fda3af9cfd1d ErrorCode=500 Ended create Csar for component Sap 2021-08-11T16:17:50.536Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Sap" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "9a01a61b-4650-491b-8d3a-3f65c8aba7e8" INVARIANT_UUID = "2e85475e-9148-4a21-806e-aae059385867" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "ff58d4ec-6a95-4413-bc95-e123bc65cfb8.assettoscatemplate" CURR_ARTIFACT_UUID = "ff58d4ec-6a95-4413-bc95-e123bc65cfb8.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Sap-template.yml,0,ff58d4ec-6a95-4413-bc95-e123bc65cfb8.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d8a95055-e2cd-4fa9-b33e-c8fe370b3610 TargetServiceName=null EntryTimestamp=2021-08-11T16:17:45.889Z InvocationID=b0e4afaa-fd53-4980-9374-fda3af9cfd1d ErrorCode=500 lock resource with id ff58d4ec-6a95-4413-bc95-e123bc65cfb8 2021-08-11T16:17:52.770Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Sap" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "9a01a61b-4650-491b-8d3a-3f65c8aba7e8" INVARIANT_UUID = "2e85475e-9148-4a21-806e-aae059385867" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "ff58d4ec-6a95-4413-bc95-e123bc65cfb8.assettoscacsar" CURR_ARTIFACT_UUID = "ff58d4ec-6a95-4413-bc95-e123bc65cfb8.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Sap-csar.csar,0,ff58d4ec-6a95-4413-bc95-e123bc65cfb8.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=6881 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=f4ac7695-a2f2-413d-b29e-e994666cd4a9 TargetServiceName=null EntryTimestamp=2021-08-11T16:17:45.889Z InvocationID=b0e4afaa-fd53-4980-9374-fda3af9cfd1d ErrorCode=500 LogTimestamp=2021-08-11T16:17:52.770105Z ResponseDesc=Created Exiting. 2021-08-11T16:17:52.771Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:17:52.869Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=01c3a60c-3387-4dd3-bede-7bdac048ec5c InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:17:52.868720Z InvocationID=d8fb717e-685c-48fb-8498-4f7cc87c5671 LogTimestamp=2021-08-11T16:17:52.868777Z Entering 2021-08-11T16:17:52.869Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=01c3a60c-3387-4dd3-bede-7bdac048ec5c InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:17:52.868720Z InvocationID=d8fb717e-685c-48fb-8498-4f7cc87c5671 LogTimestamp=2021-08-11T16:17:52.868777Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:17:53.278Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb01ea5f-9a7e-4ff6-b2ca-2991f66d8ef3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:17:53.279Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb01ea5f-9a7e-4ff6-b2ca-2991f66d8ef3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6f122261 2021-08-11T16:17:53.279Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb01ea5f-9a7e-4ff6-b2ca-2991f66d8ef3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:17:53.279Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb01ea5f-9a7e-4ff6-b2ca-2991f66d8ef3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:17:55.066Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:52.869Z InvocationID=d94b64db-1d87-4eed-b685-5338bb17c19d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:17:55.066Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:52.869Z InvocationID=d94b64db-1d87-4eed-b685-5338bb17c19d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:17:55.066Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:52.869Z InvocationID=d94b64db-1d87-4eed-b685-5338bb17c19d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:17:55.066Z [qtp2045766957-15] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:52.869Z InvocationID=d94b64db-1d87-4eed-b685-5338bb17c19d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:17:55.246Z [qtp2045766957-15] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:17:52.869Z InvocationID=d94b64db-1d87-4eed-b685-5338bb17c19d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Nfppositionelement 2021-08-11T16:17:56.162Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76a3b653-5a14-44f3-a80b-3637ea4c5d90 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:17:56.162Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76a3b653-5a14-44f3-a80b-3637ea4c5d90 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@68a45cd3 2021-08-11T16:17:56.162Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76a3b653-5a14-44f3-a80b-3637ea4c5d90 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:17:56.162Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76a3b653-5a14-44f3-a80b-3637ea4c5d90 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:17:57.379Z [qtp2045766957-15] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NfpPositionElement" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "9425fb71-85db-4038-b418-7e283294594b" INVARIANT_UUID = "540b45ad-d3f4-4f31-ad7a-1547cd8647b9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NfpPositionElement" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9425fb71-85db-4038-b418-7e283294594b EntryTimestamp=2021-08-11T16:17:52.869Z InvocationID=d94b64db-1d87-4eed-b685-5338bb17c19d PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:17:57.380Z [qtp2045766957-15] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NfpPositionElement" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "9425fb71-85db-4038-b418-7e283294594b" INVARIANT_UUID = "540b45ad-d3f4-4f31-ad7a-1547cd8647b9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NfpPositionElement" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9425fb71-85db-4038-b418-7e283294594b TargetServiceName=null EntryTimestamp=2021-08-11T16:17:52.869Z InvocationID=d94b64db-1d87-4eed-b685-5338bb17c19d ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:17:58.277Z [qtp2045766957-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NfpPositionElement" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "9425fb71-85db-4038-b418-7e283294594b" INVARIANT_UUID = "540b45ad-d3f4-4f31-ad7a-1547cd8647b9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NfpPositionElement" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9425fb71-85db-4038-b418-7e283294594b TargetServiceName=null EntryTimestamp=2021-08-11T16:17:52.869Z InvocationID=d94b64db-1d87-4eed-b685-5338bb17c19d ErrorCode=500 lock resource with id a694cb4e-7e8c-497f-b1e1-8281a64ebd50 2021-08-11T16:17:59.471Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66d81360-c245-4964-add4-b11ef7ec3482 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:17:59.471Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66d81360-c245-4964-add4-b11ef7ec3482 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@20c7ded7 2021-08-11T16:17:59.471Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66d81360-c245-4964-add4-b11ef7ec3482 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:17:59.471Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66d81360-c245-4964-add4-b11ef7ec3482 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:18:00.494Z [qtp2045766957-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NfpPositionElement" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "9425fb71-85db-4038-b418-7e283294594b" INVARIANT_UUID = "540b45ad-d3f4-4f31-ad7a-1547cd8647b9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a694cb4e-7e8c-497f-b1e1-8281a64ebd50.assettoscatemplate" CURR_ARTIFACT_UUID = "a694cb4e-7e8c-497f-b1e1-8281a64ebd50.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Nfppositionelement-template.yml,0,a694cb4e-7e8c-497f-b1e1-8281a64ebd50.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=21afe945-ad4e-421d-9dfd-388be0c06aa5 TargetServiceName=null EntryTimestamp=2021-08-11T16:17:52.869Z InvocationID=d94b64db-1d87-4eed-b685-5338bb17c19d ErrorCode=500 Starting to create Csar for component NfpPositionElement 2021-08-11T16:18:00.696Z [qtp2045766957-15] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NfpPositionElement" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "9425fb71-85db-4038-b418-7e283294594b" INVARIANT_UUID = "540b45ad-d3f4-4f31-ad7a-1547cd8647b9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a694cb4e-7e8c-497f-b1e1-8281a64ebd50.assettoscatemplate" CURR_ARTIFACT_UUID = "a694cb4e-7e8c-497f-b1e1-8281a64ebd50.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Nfppositionelement-template.yml,0,a694cb4e-7e8c-497f-b1e1-8281a64ebd50.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=21afe945-ad4e-421d-9dfd-388be0c06aa5 TargetServiceName=null EntryTimestamp=2021-08-11T16:17:52.869Z InvocationID=d94b64db-1d87-4eed-b685-5338bb17c19d ErrorCode=500 Ended create Csar for component NfpPositionElement 2021-08-11T16:18:00.696Z [qtp2045766957-15] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NfpPositionElement" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "9425fb71-85db-4038-b418-7e283294594b" INVARIANT_UUID = "540b45ad-d3f4-4f31-ad7a-1547cd8647b9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a694cb4e-7e8c-497f-b1e1-8281a64ebd50.assettoscatemplate" CURR_ARTIFACT_UUID = "a694cb4e-7e8c-497f-b1e1-8281a64ebd50.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Nfppositionelement-template.yml,0,a694cb4e-7e8c-497f-b1e1-8281a64ebd50.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=21afe945-ad4e-421d-9dfd-388be0c06aa5 TargetServiceName=null EntryTimestamp=2021-08-11T16:17:52.869Z InvocationID=d94b64db-1d87-4eed-b685-5338bb17c19d ErrorCode=500 lock resource with id a694cb4e-7e8c-497f-b1e1-8281a64ebd50 2021-08-11T16:18:01.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9835a61-f949-4cec-83b9-028db4cf9817 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:18:01.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9835a61-f949-4cec-83b9-028db4cf9817 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@34e1f556 2021-08-11T16:18:01.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9835a61-f949-4cec-83b9-028db4cf9817 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:18:01.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9835a61-f949-4cec-83b9-028db4cf9817 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:18:02.389Z [qtp2045766957-15] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NfpPositionElement" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "9425fb71-85db-4038-b418-7e283294594b" INVARIANT_UUID = "540b45ad-d3f4-4f31-ad7a-1547cd8647b9" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "a694cb4e-7e8c-497f-b1e1-8281a64ebd50.assettoscacsar" CURR_ARTIFACT_UUID = "a694cb4e-7e8c-497f-b1e1-8281a64ebd50.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Nfppositionelement-csar.csar,0,a694cb4e-7e8c-497f-b1e1-8281a64ebd50.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=9519 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=97375663-f8fe-4dc0-91d6-4a4ffdb7f6f5 TargetServiceName=null EntryTimestamp=2021-08-11T16:17:52.869Z InvocationID=d94b64db-1d87-4eed-b685-5338bb17c19d ErrorCode=500 LogTimestamp=2021-08-11T16:18:02.388785Z ResponseDesc=Created Exiting. 2021-08-11T16:18:02.389Z [qtp2045766957-15] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:18:02.409Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=6f22c403-0c68-4767-8cfb-6776c7eef8b4 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:18:02.409384Z InvocationID=57d87aec-1933-4385-8f11-da4e81557d2f LogTimestamp=2021-08-11T16:18:02.409419Z Entering 2021-08-11T16:18:02.409Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=6f22c403-0c68-4767-8cfb-6776c7eef8b4 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:18:02.409384Z InvocationID=57d87aec-1933-4385-8f11-da4e81557d2f LogTimestamp=2021-08-11T16:18:02.409419Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:18:02.644Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:18:02.409Z InvocationID=275d8dd7-8c71-4486-b239-52d936bb8c50 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:18:02.644Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:18:02.409Z InvocationID=275d8dd7-8c71-4486-b239-52d936bb8c50 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:18:02.644Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:18:02.409Z InvocationID=275d8dd7-8c71-4486-b239-52d936bb8c50 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:18:02.644Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:18:02.409Z InvocationID=275d8dd7-8c71-4486-b239-52d936bb8c50 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:18:02.645Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:18:02.409Z InvocationID=275d8dd7-8c71-4486-b239-52d936bb8c50 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Nfpposition 2021-08-11T16:18:03.305Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:18:03.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:18:03.896Z [qtp2045766957-12] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NfpPosition" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "7a9d3888-f1bd-4770-bc26-61e9071668cf" INVARIANT_UUID = "78b0e644-96ef-4710-9e8f-c0d93bbde9f3" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NfpPosition" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7a9d3888-f1bd-4770-bc26-61e9071668cf EntryTimestamp=2021-08-11T16:18:02.409Z InvocationID=275d8dd7-8c71-4486-b239-52d936bb8c50 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:18:03.896Z [qtp2045766957-12] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NfpPosition" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "7a9d3888-f1bd-4770-bc26-61e9071668cf" INVARIANT_UUID = "78b0e644-96ef-4710-9e8f-c0d93bbde9f3" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NfpPosition" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7a9d3888-f1bd-4770-bc26-61e9071668cf TargetServiceName=null EntryTimestamp=2021-08-11T16:18:02.409Z InvocationID=275d8dd7-8c71-4486-b239-52d936bb8c50 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:18:04.244Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:18:04.249Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NfpPosition" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "7a9d3888-f1bd-4770-bc26-61e9071668cf" INVARIANT_UUID = "78b0e644-96ef-4710-9e8f-c0d93bbde9f3" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NfpPosition" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7a9d3888-f1bd-4770-bc26-61e9071668cf TargetServiceName=null EntryTimestamp=2021-08-11T16:18:02.409Z InvocationID=275d8dd7-8c71-4486-b239-52d936bb8c50 ErrorCode=500 lock resource with id 91652bca-2c9a-4010-8e8a-02e0cfcd4e9b 2021-08-11T16:18:04.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04835798-3d27-4d02-93b3-af66d061286a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:18:04.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04835798-3d27-4d02-93b3-af66d061286a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4a1ec881 2021-08-11T16:18:04.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04835798-3d27-4d02-93b3-af66d061286a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:18:04.611Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04835798-3d27-4d02-93b3-af66d061286a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:18:05.861Z [qtp2045766957-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NfpPosition" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "7a9d3888-f1bd-4770-bc26-61e9071668cf" INVARIANT_UUID = "78b0e644-96ef-4710-9e8f-c0d93bbde9f3" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "91652bca-2c9a-4010-8e8a-02e0cfcd4e9b.assettoscatemplate" CURR_ARTIFACT_UUID = "91652bca-2c9a-4010-8e8a-02e0cfcd4e9b.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Nfpposition-template.yml,0,91652bca-2c9a-4010-8e8a-02e0cfcd4e9b.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ef452722-67e6-441e-bf6d-7adfd726fc98 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:02.409Z InvocationID=275d8dd7-8c71-4486-b239-52d936bb8c50 ErrorCode=500 Starting to create Csar for component NfpPosition 2021-08-11T16:18:06.005Z [qtp2045766957-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NfpPosition" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "7a9d3888-f1bd-4770-bc26-61e9071668cf" INVARIANT_UUID = "78b0e644-96ef-4710-9e8f-c0d93bbde9f3" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "91652bca-2c9a-4010-8e8a-02e0cfcd4e9b.assettoscatemplate" CURR_ARTIFACT_UUID = "91652bca-2c9a-4010-8e8a-02e0cfcd4e9b.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Nfpposition-template.yml,0,91652bca-2c9a-4010-8e8a-02e0cfcd4e9b.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ef452722-67e6-441e-bf6d-7adfd726fc98 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:02.409Z InvocationID=275d8dd7-8c71-4486-b239-52d936bb8c50 ErrorCode=500 Ended create Csar for component NfpPosition 2021-08-11T16:18:06.005Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NfpPosition" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "7a9d3888-f1bd-4770-bc26-61e9071668cf" INVARIANT_UUID = "78b0e644-96ef-4710-9e8f-c0d93bbde9f3" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "91652bca-2c9a-4010-8e8a-02e0cfcd4e9b.assettoscatemplate" CURR_ARTIFACT_UUID = "91652bca-2c9a-4010-8e8a-02e0cfcd4e9b.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Nfpposition-template.yml,0,91652bca-2c9a-4010-8e8a-02e0cfcd4e9b.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ef452722-67e6-441e-bf6d-7adfd726fc98 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:02.409Z InvocationID=275d8dd7-8c71-4486-b239-52d936bb8c50 ErrorCode=500 lock resource with id 91652bca-2c9a-4010-8e8a-02e0cfcd4e9b 2021-08-11T16:18:07.628Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NfpPosition" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "7a9d3888-f1bd-4770-bc26-61e9071668cf" INVARIANT_UUID = "78b0e644-96ef-4710-9e8f-c0d93bbde9f3" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "91652bca-2c9a-4010-8e8a-02e0cfcd4e9b.assettoscacsar" CURR_ARTIFACT_UUID = "91652bca-2c9a-4010-8e8a-02e0cfcd4e9b.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Nfpposition-csar.csar,0,91652bca-2c9a-4010-8e8a-02e0cfcd4e9b.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5219 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=2d2ee3bb-feb3-49e1-9ec9-fdcd23c29fa7 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:02.409Z InvocationID=275d8dd7-8c71-4486-b239-52d936bb8c50 ErrorCode=500 LogTimestamp=2021-08-11T16:18:07.628345Z ResponseDesc=Created Exiting. 2021-08-11T16:18:07.629Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:18:07.668Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=f737358b-1b8b-40ec-ae05-50c3e5664ea3 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:18:07.668343Z InvocationID=95de96fe-538a-4bfb-b7d2-3e98bb8903c1 LogTimestamp=2021-08-11T16:18:07.668389Z Entering 2021-08-11T16:18:07.668Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=f737358b-1b8b-40ec-ae05-50c3e5664ea3 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:18:07.668343Z InvocationID=95de96fe-538a-4bfb-b7d2-3e98bb8903c1 LogTimestamp=2021-08-11T16:18:07.668389Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:18:07.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6baf3abe-d08e-4b73-8aae-1e501d5c52e3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:18:07.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6baf3abe-d08e-4b73-8aae-1e501d5c52e3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@9647d5 2021-08-11T16:18:07.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6baf3abe-d08e-4b73-8aae-1e501d5c52e3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:18:07.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6baf3abe-d08e-4b73-8aae-1e501d5c52e3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:18:08.096Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:18:07.669Z InvocationID=a17df887-1ed5-4878-acde-f1be66cbf38b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:18:08.096Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:18:07.669Z InvocationID=a17df887-1ed5-4878-acde-f1be66cbf38b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:18:08.096Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:18:07.669Z InvocationID=a17df887-1ed5-4878-acde-f1be66cbf38b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:18:08.096Z [qtp2045766957-17] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:18:07.669Z InvocationID=a17df887-1ed5-4878-acde-f1be66cbf38b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:18:08.112Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:18:07.669Z InvocationID=a17df887-1ed5-4878-acde-f1be66cbf38b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Nfp 2021-08-11T16:18:09.377Z [qtp2045766957-17] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NFP" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "c5304cdc-cd32-4eac-88f2-fb4281105c6c" INVARIANT_UUID = "6229a26c-5f32-4fb5-814d-95b669f0ef49" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NFP" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c5304cdc-cd32-4eac-88f2-fb4281105c6c EntryTimestamp=2021-08-11T16:18:07.669Z InvocationID=a17df887-1ed5-4878-acde-f1be66cbf38b PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:18:09.377Z [qtp2045766957-17] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NFP" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "c5304cdc-cd32-4eac-88f2-fb4281105c6c" INVARIANT_UUID = "6229a26c-5f32-4fb5-814d-95b669f0ef49" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NFP" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c5304cdc-cd32-4eac-88f2-fb4281105c6c TargetServiceName=null EntryTimestamp=2021-08-11T16:18:07.669Z InvocationID=a17df887-1ed5-4878-acde-f1be66cbf38b ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:18:09.950Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NFP" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "c5304cdc-cd32-4eac-88f2-fb4281105c6c" INVARIANT_UUID = "6229a26c-5f32-4fb5-814d-95b669f0ef49" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NFP" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c5304cdc-cd32-4eac-88f2-fb4281105c6c TargetServiceName=null EntryTimestamp=2021-08-11T16:18:07.669Z InvocationID=a17df887-1ed5-4878-acde-f1be66cbf38b ErrorCode=500 lock resource with id 8ccb3b09-4a78-4251-9971-c6c3d140afc2 2021-08-11T16:18:10.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9283618b-4c2f-4cf4-8cd0-4ed3cacd660a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:18:10.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9283618b-4c2f-4cf4-8cd0-4ed3cacd660a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1e390076 2021-08-11T16:18:10.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9283618b-4c2f-4cf4-8cd0-4ed3cacd660a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:18:10.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9283618b-4c2f-4cf4-8cd0-4ed3cacd660a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:18:11.557Z [qtp2045766957-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NFP" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "c5304cdc-cd32-4eac-88f2-fb4281105c6c" INVARIANT_UUID = "6229a26c-5f32-4fb5-814d-95b669f0ef49" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "8ccb3b09-4a78-4251-9971-c6c3d140afc2.assettoscatemplate" CURR_ARTIFACT_UUID = "8ccb3b09-4a78-4251-9971-c6c3d140afc2.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Nfp-template.yml,0,8ccb3b09-4a78-4251-9971-c6c3d140afc2.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=5054da93-fac7-4a18-9021-01f1d3a96445 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:07.669Z InvocationID=a17df887-1ed5-4878-acde-f1be66cbf38b ErrorCode=500 Starting to create Csar for component NFP 2021-08-11T16:18:11.684Z [qtp2045766957-17] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NFP" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "c5304cdc-cd32-4eac-88f2-fb4281105c6c" INVARIANT_UUID = "6229a26c-5f32-4fb5-814d-95b669f0ef49" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "8ccb3b09-4a78-4251-9971-c6c3d140afc2.assettoscatemplate" CURR_ARTIFACT_UUID = "8ccb3b09-4a78-4251-9971-c6c3d140afc2.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Nfp-template.yml,0,8ccb3b09-4a78-4251-9971-c6c3d140afc2.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=5054da93-fac7-4a18-9021-01f1d3a96445 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:07.669Z InvocationID=a17df887-1ed5-4878-acde-f1be66cbf38b ErrorCode=500 Ended create Csar for component NFP 2021-08-11T16:18:11.684Z [qtp2045766957-17] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NFP" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "c5304cdc-cd32-4eac-88f2-fb4281105c6c" INVARIANT_UUID = "6229a26c-5f32-4fb5-814d-95b669f0ef49" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "8ccb3b09-4a78-4251-9971-c6c3d140afc2.assettoscatemplate" CURR_ARTIFACT_UUID = "8ccb3b09-4a78-4251-9971-c6c3d140afc2.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Nfp-template.yml,0,8ccb3b09-4a78-4251-9971-c6c3d140afc2.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=5054da93-fac7-4a18-9021-01f1d3a96445 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:07.669Z InvocationID=a17df887-1ed5-4878-acde-f1be66cbf38b ErrorCode=500 lock resource with id 8ccb3b09-4a78-4251-9971-c6c3d140afc2 2021-08-11T16:18:13.326Z [qtp2045766957-17] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NFP" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "c5304cdc-cd32-4eac-88f2-fb4281105c6c" INVARIANT_UUID = "6229a26c-5f32-4fb5-814d-95b669f0ef49" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "8ccb3b09-4a78-4251-9971-c6c3d140afc2.assettoscacsar" CURR_ARTIFACT_UUID = "8ccb3b09-4a78-4251-9971-c6c3d140afc2.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Nfp-csar.csar,0,8ccb3b09-4a78-4251-9971-c6c3d140afc2.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5656 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=eb6205c2-c96b-4ad9-a64b-94d0fad9e813 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:07.669Z InvocationID=a17df887-1ed5-4878-acde-f1be66cbf38b ErrorCode=500 LogTimestamp=2021-08-11T16:18:13.325812Z ResponseDesc=Created Exiting. 2021-08-11T16:18:13.327Z [qtp2045766957-17] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:18:13.349Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=c8b1cc19-e9e6-4754-8c49-91b4757239a5 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:18:13.348967Z InvocationID=2adc7370-62dd-48be-a7e0-d6c335d8dee3 LogTimestamp=2021-08-11T16:18:13.349015Z Entering 2021-08-11T16:18:13.349Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=c8b1cc19-e9e6-4754-8c49-91b4757239a5 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:18:13.348967Z InvocationID=2adc7370-62dd-48be-a7e0-d6c335d8dee3 LogTimestamp=2021-08-11T16:18:13.349015Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:18:13.698Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b031ccba-b632-4e52-bb24-7350e4a5565d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:18:13.698Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b031ccba-b632-4e52-bb24-7350e4a5565d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@25c90da2 2021-08-11T16:18:13.698Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b031ccba-b632-4e52-bb24-7350e4a5565d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:18:13.698Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b031ccba-b632-4e52-bb24-7350e4a5565d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:18:13.949Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:18:13.349Z InvocationID=4bbece78-cb0b-4470-8dac-84497bd21b21 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:18:13.949Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:18:13.349Z InvocationID=4bbece78-cb0b-4470-8dac-84497bd21b21 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:18:13.949Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:18:13.349Z InvocationID=4bbece78-cb0b-4470-8dac-84497bd21b21 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:18:13.949Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:18:13.349Z InvocationID=4bbece78-cb0b-4470-8dac-84497bd21b21 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:18:13.999Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:18:13.349Z InvocationID=4bbece78-cb0b-4470-8dac-84497bd21b21 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name Forwarding 2021-08-11T16:18:15.291Z [qtp2045766957-12] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Forwarding" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "550af884-897d-4ff0-b3ea-f772971ccda7" INVARIANT_UUID = "0f5bc309-c6ca-4d3c-8527-6a32b0ae068d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Forwarding" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=550af884-897d-4ff0-b3ea-f772971ccda7 EntryTimestamp=2021-08-11T16:18:13.349Z InvocationID=4bbece78-cb0b-4470-8dac-84497bd21b21 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:18:15.292Z [qtp2045766957-12] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Forwarding" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "550af884-897d-4ff0-b3ea-f772971ccda7" INVARIANT_UUID = "0f5bc309-c6ca-4d3c-8527-6a32b0ae068d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Forwarding" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=550af884-897d-4ff0-b3ea-f772971ccda7 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:13.349Z InvocationID=4bbece78-cb0b-4470-8dac-84497bd21b21 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:18:15.571Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Forwarding" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "550af884-897d-4ff0-b3ea-f772971ccda7" INVARIANT_UUID = "0f5bc309-c6ca-4d3c-8527-6a32b0ae068d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Forwarding" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=550af884-897d-4ff0-b3ea-f772971ccda7 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:13.349Z InvocationID=4bbece78-cb0b-4470-8dac-84497bd21b21 ErrorCode=500 lock resource with id 0289e540-97ee-444e-8853-d3897bcc2f73 2021-08-11T16:18:16.703Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41f382b8-4173-4609-910e-8f2a9de5cc23 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:18:16.703Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41f382b8-4173-4609-910e-8f2a9de5cc23 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@192f835a 2021-08-11T16:18:16.703Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41f382b8-4173-4609-910e-8f2a9de5cc23 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:18:16.703Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41f382b8-4173-4609-910e-8f2a9de5cc23 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:18:17.235Z [qtp2045766957-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Forwarding" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "550af884-897d-4ff0-b3ea-f772971ccda7" INVARIANT_UUID = "0f5bc309-c6ca-4d3c-8527-6a32b0ae068d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "0289e540-97ee-444e-8853-d3897bcc2f73.assettoscatemplate" CURR_ARTIFACT_UUID = "0289e540-97ee-444e-8853-d3897bcc2f73.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Forwarding-template.yml,0,0289e540-97ee-444e-8853-d3897bcc2f73.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=28b1f4e5-8292-442b-924d-7bbe7655de62 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:13.349Z InvocationID=4bbece78-cb0b-4470-8dac-84497bd21b21 ErrorCode=500 Starting to create Csar for component Forwarding 2021-08-11T16:18:17.408Z [qtp2045766957-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Forwarding" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "550af884-897d-4ff0-b3ea-f772971ccda7" INVARIANT_UUID = "0f5bc309-c6ca-4d3c-8527-6a32b0ae068d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "0289e540-97ee-444e-8853-d3897bcc2f73.assettoscatemplate" CURR_ARTIFACT_UUID = "0289e540-97ee-444e-8853-d3897bcc2f73.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Forwarding-template.yml,0,0289e540-97ee-444e-8853-d3897bcc2f73.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=28b1f4e5-8292-442b-924d-7bbe7655de62 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:13.349Z InvocationID=4bbece78-cb0b-4470-8dac-84497bd21b21 ErrorCode=500 Ended create Csar for component Forwarding 2021-08-11T16:18:17.409Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Forwarding" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "550af884-897d-4ff0-b3ea-f772971ccda7" INVARIANT_UUID = "0f5bc309-c6ca-4d3c-8527-6a32b0ae068d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "0289e540-97ee-444e-8853-d3897bcc2f73.assettoscatemplate" CURR_ARTIFACT_UUID = "0289e540-97ee-444e-8853-d3897bcc2f73.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Forwarding-template.yml,0,0289e540-97ee-444e-8853-d3897bcc2f73.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=28b1f4e5-8292-442b-924d-7bbe7655de62 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:13.349Z InvocationID=4bbece78-cb0b-4470-8dac-84497bd21b21 ErrorCode=500 lock resource with id 0289e540-97ee-444e-8853-d3897bcc2f73 2021-08-11T16:18:18.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:18:18.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:18:19.028Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Forwarding" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "550af884-897d-4ff0-b3ea-f772971ccda7" INVARIANT_UUID = "0f5bc309-c6ca-4d3c-8527-6a32b0ae068d" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "0289e540-97ee-444e-8853-d3897bcc2f73.assettoscacsar" CURR_ARTIFACT_UUID = "0289e540-97ee-444e-8853-d3897bcc2f73.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Forwarding-csar.csar,0,0289e540-97ee-444e-8853-d3897bcc2f73.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5679 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c994499d-70b2-4274-b26e-3b8d4bdad9ca TargetServiceName=null EntryTimestamp=2021-08-11T16:18:13.349Z InvocationID=4bbece78-cb0b-4470-8dac-84497bd21b21 ErrorCode=500 LogTimestamp=2021-08-11T16:18:19.028393Z ResponseDesc=Created Exiting. 2021-08-11T16:18:19.029Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:18:19.046Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=47503555-78bc-4980-a509-d53fd56fcf95 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:18:19.046071Z InvocationID=41e26689-3649-437e-95b0-1e3a05c47ed9 LogTimestamp=2021-08-11T16:18:19.046114Z Entering 2021-08-11T16:18:19.046Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=47503555-78bc-4980-a509-d53fd56fcf95 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:18:19.046071Z InvocationID=41e26689-3649-437e-95b0-1e3a05c47ed9 LogTimestamp=2021-08-11T16:18:19.046114Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:18:19.377Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:18:19.046Z InvocationID=0773dd1e-cdfd-42e2-a0af-775ca5cee3a9 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id 05b4e29f-606e-40e5-a4fd-bafd6aeb3b7a 2021-08-11T16:18:19.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96f2f30f-3620-41a0-b49a-63fcbbd840b3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:18:19.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96f2f30f-3620-41a0-b49a-63fcbbd840b3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2bb5c745 2021-08-11T16:18:19.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96f2f30f-3620-41a0-b49a-63fcbbd840b3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:18:19.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96f2f30f-3620-41a0-b49a-63fcbbd840b3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:18:20.029Z [qtp2045766957-16] INFO o.o.s.b.c.l.LifecycleBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:18:19.046Z InvocationID=0773dd1e-cdfd-42e2-a0af-775ca5cee3a9 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p owner of resource 05b4e29f-606e-40e5-a4fd-bafd6aeb3b7a is jh0003 2021-08-11T16:18:20.151Z [qtp2045766957-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "5c6e1870-9543-4bb9-8978-b675ef31d566" INVARIANT_UUID = "a9e1a9b1-f8e2-48e8-89d4-80b004df2b96" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:18:19.046Z InvocationID=0773dd1e-cdfd-42e2-a0af-775ca5cee3a9 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:18:20.152Z [qtp2045766957-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "5c6e1870-9543-4bb9-8978-b675ef31d566" INVARIANT_UUID = "a9e1a9b1-f8e2-48e8-89d4-80b004df2b96" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2021-08-11T16:18:19.046Z InvocationID=0773dd1e-cdfd-42e2-a0af-775ca5cee3a9 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:18:20.578Z [qtp2045766957-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "5c6e1870-9543-4bb9-8978-b675ef31d566" INVARIANT_UUID = "a9e1a9b1-f8e2-48e8-89d4-80b004df2b96" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d78fdba9-12dc-4f14-9aad-e32c352705cb TargetServiceName=null EntryTimestamp=2021-08-11T16:18:19.046Z InvocationID=0773dd1e-cdfd-42e2-a0af-775ca5cee3a9 ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:18:20.579Z [qtp2045766957-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "5c6e1870-9543-4bb9-8978-b675ef31d566" INVARIANT_UUID = "a9e1a9b1-f8e2-48e8-89d4-80b004df2b96" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d78fdba9-12dc-4f14-9aad-e32c352705cb TargetServiceName=null EntryTimestamp=2021-08-11T16:18:19.046Z InvocationID=0773dd1e-cdfd-42e2-a0af-775ca5cee3a9 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:18:21.858Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "5c6e1870-9543-4bb9-8978-b675ef31d566" INVARIANT_UUID = "a9e1a9b1-f8e2-48e8-89d4-80b004df2b96" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.Compute" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d78fdba9-12dc-4f14-9aad-e32c352705cb TargetServiceName=null EntryTimestamp=2021-08-11T16:18:19.046Z InvocationID=0773dd1e-cdfd-42e2-a0af-775ca5cee3a9 ErrorCode=500 lock resource with id 188d675b-75e2-4b45-97f9-894acaf6b3d1 2021-08-11T16:18:22.562Z [qtp2045766957-16] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "5c6e1870-9543-4bb9-8978-b675ef31d566" INVARIANT_UUID = "a9e1a9b1-f8e2-48e8-89d4-80b004df2b96" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.Compute" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d78fdba9-12dc-4f14-9aad-e32c352705cb TargetServiceName=null EntryTimestamp=2021-08-11T16:18:19.046Z InvocationID=0773dd1e-cdfd-42e2-a0af-775ca5cee3a9 ErrorCode=500 owner of resource 188d675b-75e2-4b45-97f9-894acaf6b3d1 is jh0003 2021-08-11T16:18:22.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f219e1f-03a7-4607-b5fb-9c3c324a9045 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:18:22.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f219e1f-03a7-4607-b5fb-9c3c324a9045 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7d5948f5 2021-08-11T16:18:22.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f219e1f-03a7-4607-b5fb-9c3c324a9045 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:18:22.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f219e1f-03a7-4607-b5fb-9c3c324a9045 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:18:23.491Z [qtp2045766957-16] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d78fdba9-12dc-4f14-9aad-e32c352705cb" INVARIANT_UUID = "a9e1a9b1-f8e2-48e8-89d4-80b004df2b96" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d78fdba9-12dc-4f14-9aad-e32c352705cb TargetServiceName=null EntryTimestamp=2021-08-11T16:18:19.046Z InvocationID=0773dd1e-cdfd-42e2-a0af-775ca5cee3a9 ErrorCode=500 owner of resource 188d675b-75e2-4b45-97f9-894acaf6b3d1 is jh0003 2021-08-11T16:18:23.491Z [qtp2045766957-16] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d78fdba9-12dc-4f14-9aad-e32c352705cb" INVARIANT_UUID = "a9e1a9b1-f8e2-48e8-89d4-80b004df2b96" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d78fdba9-12dc-4f14-9aad-e32c352705cb TargetServiceName=null EntryTimestamp=2021-08-11T16:18:19.046Z InvocationID=0773dd1e-cdfd-42e2-a0af-775ca5cee3a9 ErrorCode=500 validate before certification change. resource name=VDU Compute, oldState=NOT_CERTIFIED_CHECKIN, owner userId=jh0003 2021-08-11T16:18:23.491Z [qtp2045766957-16] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d78fdba9-12dc-4f14-9aad-e32c352705cb" INVARIANT_UUID = "a9e1a9b1-f8e2-48e8-89d4-80b004df2b96" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d78fdba9-12dc-4f14-9aad-e32c352705cb TargetServiceName=null EntryTimestamp=2021-08-11T16:18:19.046Z InvocationID=0773dd1e-cdfd-42e2-a0af-775ca5cee3a9 ErrorCode=500 start performing certification change for resource 188d675b-75e2-4b45-97f9-894acaf6b3d1 2021-08-11T16:18:23.921Z [qtp2045766957-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "d78fdba9-12dc-4f14-9aad-e32c352705cb" INVARIANT_UUID = "a9e1a9b1-f8e2-48e8-89d4-80b004df2b96" PREV_VERSION = "" CURR_VERSION = "1.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "05b4e29f-606e-40e5-a4fd-bafd6aeb3b7a.assettoscatemplate" CURR_ARTIFACT_UUID = "05b4e29f-606e-40e5-a4fd-bafd6aeb3b7a.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduCompute-template.yml,0,05b4e29f-606e-40e5-a4fd-bafd6aeb3b7a.assettoscatemplate,1,b508aff1-6062-4b17-bfca-df414af329c8" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b29594cd-edf8-48ff-9c1b-1b8733e18ccc TargetServiceName=null EntryTimestamp=2021-08-11T16:18:19.046Z InvocationID=0773dd1e-cdfd-42e2-a0af-775ca5cee3a9 ErrorCode=500 Starting to create Csar for component VDU Compute 2021-08-11T16:18:24.089Z [qtp2045766957-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "d78fdba9-12dc-4f14-9aad-e32c352705cb" INVARIANT_UUID = "a9e1a9b1-f8e2-48e8-89d4-80b004df2b96" PREV_VERSION = "" CURR_VERSION = "1.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "05b4e29f-606e-40e5-a4fd-bafd6aeb3b7a.assettoscatemplate" CURR_ARTIFACT_UUID = "05b4e29f-606e-40e5-a4fd-bafd6aeb3b7a.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduCompute-template.yml,0,05b4e29f-606e-40e5-a4fd-bafd6aeb3b7a.assettoscatemplate,1,b508aff1-6062-4b17-bfca-df414af329c8" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b29594cd-edf8-48ff-9c1b-1b8733e18ccc TargetServiceName=null EntryTimestamp=2021-08-11T16:18:19.046Z InvocationID=0773dd1e-cdfd-42e2-a0af-775ca5cee3a9 ErrorCode=500 Ended create Csar for component VDU Compute 2021-08-11T16:18:25.079Z [qtp2045766957-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d78fdba9-12dc-4f14-9aad-e32c352705cb" INVARIANT_UUID = "a9e1a9b1-f8e2-48e8-89d4-80b004df2b96" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e02cdcbc-7c93-49ac-862f-ceea69d3fb24 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:19.046Z InvocationID=0773dd1e-cdfd-42e2-a0af-775ca5cee3a9 ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:18:25.079Z [qtp2045766957-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d78fdba9-12dc-4f14-9aad-e32c352705cb" INVARIANT_UUID = "a9e1a9b1-f8e2-48e8-89d4-80b004df2b96" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e02cdcbc-7c93-49ac-862f-ceea69d3fb24 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:19.046Z InvocationID=0773dd1e-cdfd-42e2-a0af-775ca5cee3a9 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:18:25.079Z [qtp2045766957-16] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d78fdba9-12dc-4f14-9aad-e32c352705cb" INVARIANT_UUID = "a9e1a9b1-f8e2-48e8-89d4-80b004df2b96" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e02cdcbc-7c93-49ac-862f-ceea69d3fb24 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:19.046Z InvocationID=0773dd1e-cdfd-42e2-a0af-775ca5cee3a9 ErrorCode=500 unlock component 188d675b-75e2-4b45-97f9-894acaf6b3d1 2021-08-11T16:18:25.080Z [qtp2045766957-16] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d78fdba9-12dc-4f14-9aad-e32c352705cb" INVARIANT_UUID = "a9e1a9b1-f8e2-48e8-89d4-80b004df2b96" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e02cdcbc-7c93-49ac-862f-ceea69d3fb24 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:19.046Z InvocationID=0773dd1e-cdfd-42e2-a0af-775ca5cee3a9 ErrorCode=500 During change state, another component 188d675b-75e2-4b45-97f9-894acaf6b3d1 has been created/updated 2021-08-11T16:18:25.693Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU Compute" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d78fdba9-12dc-4f14-9aad-e32c352705cb" INVARIANT_UUID = "a9e1a9b1-f8e2-48e8-89d4-80b004df2b96" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=6647 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e02cdcbc-7c93-49ac-862f-ceea69d3fb24 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:19.046Z InvocationID=0773dd1e-cdfd-42e2-a0af-775ca5cee3a9 ErrorCode=500 LogTimestamp=2021-08-11T16:18:25.693031Z ResponseDesc=Created Exiting. 2021-08-11T16:18:25.693Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:18:25.706Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=28b985ea-1142-4afc-9a01-ef01e5ca2c75 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:18:25.706180Z InvocationID=017e647c-da2d-4ea7-857c-3445e1b0654f LogTimestamp=2021-08-11T16:18:25.706212Z Entering 2021-08-11T16:18:25.706Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=28b985ea-1142-4afc-9a01-ef01e5ca2c75 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:18:25.706180Z InvocationID=017e647c-da2d-4ea7-857c-3445e1b0654f LogTimestamp=2021-08-11T16:18:25.706212Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:18:25.798Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56666557-9b6f-473d-8094-04c0967ee0b0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:18:25.798Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56666557-9b6f-473d-8094-04c0967ee0b0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1010eee1 2021-08-11T16:18:25.798Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56666557-9b6f-473d-8094-04c0967ee0b0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:18:25.798Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56666557-9b6f-473d-8094-04c0967ee0b0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:18:26.372Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:18:25.706Z InvocationID=5cd2fd0b-cdc0-4ab1-bf33-c345f3b3cc24 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id 0bd866ed-f620-48c3-889c-5bff0971b4f2 2021-08-11T16:18:27.266Z [qtp2045766957-12] INFO o.o.s.b.c.l.LifecycleBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:18:25.706Z InvocationID=5cd2fd0b-cdc0-4ab1-bf33-c345f3b3cc24 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p owner of resource 0bd866ed-f620-48c3-889c-5bff0971b4f2 is jh0003 2021-08-11T16:18:27.460Z [qtp2045766957-12] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "8c54c0a4-6e27-4ac2-aafa-99abfed1890f" INVARIANT_UUID = "6d5e9aee-a62c-4f59-a5bb-2bc9216796e4" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:18:25.706Z InvocationID=5cd2fd0b-cdc0-4ab1-bf33-c345f3b3cc24 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:18:27.461Z [qtp2045766957-12] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "8c54c0a4-6e27-4ac2-aafa-99abfed1890f" INVARIANT_UUID = "6d5e9aee-a62c-4f59-a5bb-2bc9216796e4" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2021-08-11T16:18:25.706Z InvocationID=5cd2fd0b-cdc0-4ab1-bf33-c345f3b3cc24 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:18:28.464Z [qtp2045766957-12] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "8c54c0a4-6e27-4ac2-aafa-99abfed1890f" INVARIANT_UUID = "6d5e9aee-a62c-4f59-a5bb-2bc9216796e4" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=14ecfff3-631a-4090-8d16-50d1fc5cf544 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:25.706Z InvocationID=5cd2fd0b-cdc0-4ab1-bf33-c345f3b3cc24 ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:18:28.464Z [qtp2045766957-12] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "8c54c0a4-6e27-4ac2-aafa-99abfed1890f" INVARIANT_UUID = "6d5e9aee-a62c-4f59-a5bb-2bc9216796e4" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=14ecfff3-631a-4090-8d16-50d1fc5cf544 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:25.706Z InvocationID=5cd2fd0b-cdc0-4ab1-bf33-c345f3b3cc24 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:18:28.952Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e74820b0-4df3-4265-b4ef-ffb8a55e38eb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:18:28.953Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e74820b0-4df3-4265-b4ef-ffb8a55e38eb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@75408aea 2021-08-11T16:18:28.953Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e74820b0-4df3-4265-b4ef-ffb8a55e38eb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:18:28.953Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e74820b0-4df3-4265-b4ef-ffb8a55e38eb PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:18:29.971Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "8c54c0a4-6e27-4ac2-aafa-99abfed1890f" INVARIANT_UUID = "6d5e9aee-a62c-4f59-a5bb-2bc9216796e4" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualFileStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=14ecfff3-631a-4090-8d16-50d1fc5cf544 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:25.706Z InvocationID=5cd2fd0b-cdc0-4ab1-bf33-c345f3b3cc24 ErrorCode=500 lock resource with id 6a1eebd0-5d90-4d03-8efe-457dd3e5fb74 2021-08-11T16:18:30.503Z [qtp2045766957-12] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "8c54c0a4-6e27-4ac2-aafa-99abfed1890f" INVARIANT_UUID = "6d5e9aee-a62c-4f59-a5bb-2bc9216796e4" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualFileStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=14ecfff3-631a-4090-8d16-50d1fc5cf544 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:25.706Z InvocationID=5cd2fd0b-cdc0-4ab1-bf33-c345f3b3cc24 ErrorCode=500 owner of resource 6a1eebd0-5d90-4d03-8efe-457dd3e5fb74 is jh0003 2021-08-11T16:18:31.373Z [qtp2045766957-12] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "14ecfff3-631a-4090-8d16-50d1fc5cf544" INVARIANT_UUID = "6d5e9aee-a62c-4f59-a5bb-2bc9216796e4" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=14ecfff3-631a-4090-8d16-50d1fc5cf544 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:25.706Z InvocationID=5cd2fd0b-cdc0-4ab1-bf33-c345f3b3cc24 ErrorCode=500 owner of resource 6a1eebd0-5d90-4d03-8efe-457dd3e5fb74 is jh0003 2021-08-11T16:18:31.373Z [qtp2045766957-12] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "14ecfff3-631a-4090-8d16-50d1fc5cf544" INVARIANT_UUID = "6d5e9aee-a62c-4f59-a5bb-2bc9216796e4" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=14ecfff3-631a-4090-8d16-50d1fc5cf544 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:25.706Z InvocationID=5cd2fd0b-cdc0-4ab1-bf33-c345f3b3cc24 ErrorCode=500 validate before certification change. resource name=VDU VirtualFileStorage, oldState=NOT_CERTIFIED_CHECKIN, owner userId=jh0003 2021-08-11T16:18:31.373Z [qtp2045766957-12] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "14ecfff3-631a-4090-8d16-50d1fc5cf544" INVARIANT_UUID = "6d5e9aee-a62c-4f59-a5bb-2bc9216796e4" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=14ecfff3-631a-4090-8d16-50d1fc5cf544 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:25.706Z InvocationID=5cd2fd0b-cdc0-4ab1-bf33-c345f3b3cc24 ErrorCode=500 start performing certification change for resource 6a1eebd0-5d90-4d03-8efe-457dd3e5fb74 2021-08-11T16:18:32.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=408139ba-794e-4cbb-9a40-931f0abe6fbd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:18:32.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=408139ba-794e-4cbb-9a40-931f0abe6fbd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@704ae4e8 2021-08-11T16:18:32.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=408139ba-794e-4cbb-9a40-931f0abe6fbd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:18:32.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=408139ba-794e-4cbb-9a40-931f0abe6fbd PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:18:32.305Z [qtp2045766957-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "14ecfff3-631a-4090-8d16-50d1fc5cf544" INVARIANT_UUID = "6d5e9aee-a62c-4f59-a5bb-2bc9216796e4" PREV_VERSION = "" CURR_VERSION = "1.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "0bd866ed-f620-48c3-889c-5bff0971b4f2.assettoscatemplate" CURR_ARTIFACT_UUID = "0bd866ed-f620-48c3-889c-5bff0971b4f2.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualfilestorage-template.yml,0,0bd866ed-f620-48c3-889c-5bff0971b4f2.assettoscatemplate,1,15cc45d1-d4ca-4ed6-b73d-12d82ac2bb83" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=66202be3-4c43-400c-b396-2d946027f652 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:25.706Z InvocationID=5cd2fd0b-cdc0-4ab1-bf33-c345f3b3cc24 ErrorCode=500 Starting to create Csar for component VDU VirtualFileStorage 2021-08-11T16:18:32.463Z [qtp2045766957-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "14ecfff3-631a-4090-8d16-50d1fc5cf544" INVARIANT_UUID = "6d5e9aee-a62c-4f59-a5bb-2bc9216796e4" PREV_VERSION = "" CURR_VERSION = "1.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "0bd866ed-f620-48c3-889c-5bff0971b4f2.assettoscatemplate" CURR_ARTIFACT_UUID = "0bd866ed-f620-48c3-889c-5bff0971b4f2.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualfilestorage-template.yml,0,0bd866ed-f620-48c3-889c-5bff0971b4f2.assettoscatemplate,1,15cc45d1-d4ca-4ed6-b73d-12d82ac2bb83" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=66202be3-4c43-400c-b396-2d946027f652 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:25.706Z InvocationID=5cd2fd0b-cdc0-4ab1-bf33-c345f3b3cc24 ErrorCode=500 Ended create Csar for component VDU VirtualFileStorage 2021-08-11T16:18:33.280Z [qtp2045766957-12] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "14ecfff3-631a-4090-8d16-50d1fc5cf544" INVARIANT_UUID = "6d5e9aee-a62c-4f59-a5bb-2bc9216796e4" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ca1b2570-6b4e-427e-98b8-0ba82dd12735 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:25.706Z InvocationID=5cd2fd0b-cdc0-4ab1-bf33-c345f3b3cc24 ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:18:33.281Z [qtp2045766957-12] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "14ecfff3-631a-4090-8d16-50d1fc5cf544" INVARIANT_UUID = "6d5e9aee-a62c-4f59-a5bb-2bc9216796e4" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ca1b2570-6b4e-427e-98b8-0ba82dd12735 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:25.706Z InvocationID=5cd2fd0b-cdc0-4ab1-bf33-c345f3b3cc24 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:18:33.282Z [qtp2045766957-12] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "14ecfff3-631a-4090-8d16-50d1fc5cf544" INVARIANT_UUID = "6d5e9aee-a62c-4f59-a5bb-2bc9216796e4" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ca1b2570-6b4e-427e-98b8-0ba82dd12735 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:25.706Z InvocationID=5cd2fd0b-cdc0-4ab1-bf33-c345f3b3cc24 ErrorCode=500 unlock component 6a1eebd0-5d90-4d03-8efe-457dd3e5fb74 2021-08-11T16:18:33.282Z [qtp2045766957-12] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "14ecfff3-631a-4090-8d16-50d1fc5cf544" INVARIANT_UUID = "6d5e9aee-a62c-4f59-a5bb-2bc9216796e4" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ca1b2570-6b4e-427e-98b8-0ba82dd12735 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:25.706Z InvocationID=5cd2fd0b-cdc0-4ab1-bf33-c345f3b3cc24 ErrorCode=500 During change state, another component 6a1eebd0-5d90-4d03-8efe-457dd3e5fb74 has been created/updated 2021-08-11T16:18:33.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:18:33.317Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:18:33.898Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU VirtualFileStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "14ecfff3-631a-4090-8d16-50d1fc5cf544" INVARIANT_UUID = "6d5e9aee-a62c-4f59-a5bb-2bc9216796e4" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=8192 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ca1b2570-6b4e-427e-98b8-0ba82dd12735 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:25.706Z InvocationID=5cd2fd0b-cdc0-4ab1-bf33-c345f3b3cc24 ErrorCode=500 LogTimestamp=2021-08-11T16:18:33.898149Z ResponseDesc=Created Exiting. 2021-08-11T16:18:33.899Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:18:33.914Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=17f13491-29ba-45a1-b62d-b040e7a070a3 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:18:33.913808Z InvocationID=62a29904-c25e-45ff-a0b9-464f4de6808b LogTimestamp=2021-08-11T16:18:33.913857Z Entering 2021-08-11T16:18:33.914Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=17f13491-29ba-45a1-b62d-b040e7a070a3 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:18:33.913808Z InvocationID=62a29904-c25e-45ff-a0b9-464f4de6808b LogTimestamp=2021-08-11T16:18:33.913857Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:18:34.173Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:18:33.914Z InvocationID=b8c71ccc-738f-477d-a25d-6084eb52ad6a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id cf9121f9-869a-42a4-a3b3-66aff5eeb3a3 2021-08-11T16:18:34.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e394a07-4940-4709-a618-9b6c81383eb6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:18:34.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e394a07-4940-4709-a618-9b6c81383eb6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@476304f4 2021-08-11T16:18:34.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e394a07-4940-4709-a618-9b6c81383eb6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:18:34.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e394a07-4940-4709-a618-9b6c81383eb6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:18:34.823Z [qtp2045766957-14] INFO o.o.s.b.c.l.LifecycleBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:18:33.914Z InvocationID=b8c71ccc-738f-477d-a25d-6084eb52ad6a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p owner of resource cf9121f9-869a-42a4-a3b3-66aff5eeb3a3 is jh0003 2021-08-11T16:18:34.883Z [qtp2045766957-14] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "eaf9fe82-d943-4ca2-9c7a-4066cd027155" INVARIANT_UUID = "5fce6f56-b8f4-4441-a8d6-1217f0721dfe" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:18:33.914Z InvocationID=b8c71ccc-738f-477d-a25d-6084eb52ad6a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:18:34.884Z [qtp2045766957-14] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "eaf9fe82-d943-4ca2-9c7a-4066cd027155" INVARIANT_UUID = "5fce6f56-b8f4-4441-a8d6-1217f0721dfe" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2021-08-11T16:18:33.914Z InvocationID=b8c71ccc-738f-477d-a25d-6084eb52ad6a ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:18:35.243Z [qtp2045766957-14] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "eaf9fe82-d943-4ca2-9c7a-4066cd027155" INVARIANT_UUID = "5fce6f56-b8f4-4441-a8d6-1217f0721dfe" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e945dee7-ca2a-418b-b800-a8b2d6f303f6 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:33.914Z InvocationID=b8c71ccc-738f-477d-a25d-6084eb52ad6a ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:18:35.243Z [qtp2045766957-14] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "eaf9fe82-d943-4ca2-9c7a-4066cd027155" INVARIANT_UUID = "5fce6f56-b8f4-4441-a8d6-1217f0721dfe" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e945dee7-ca2a-418b-b800-a8b2d6f303f6 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:33.914Z InvocationID=b8c71ccc-738f-477d-a25d-6084eb52ad6a ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:18:36.087Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "eaf9fe82-d943-4ca2-9c7a-4066cd027155" INVARIANT_UUID = "5fce6f56-b8f4-4441-a8d6-1217f0721dfe" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VnfExtCp" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e945dee7-ca2a-418b-b800-a8b2d6f303f6 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:33.914Z InvocationID=b8c71ccc-738f-477d-a25d-6084eb52ad6a ErrorCode=500 lock resource with id a514d9bd-7ad5-43e6-a4a1-82801846f755 2021-08-11T16:18:36.616Z [qtp2045766957-14] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "eaf9fe82-d943-4ca2-9c7a-4066cd027155" INVARIANT_UUID = "5fce6f56-b8f4-4441-a8d6-1217f0721dfe" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VnfExtCp" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e945dee7-ca2a-418b-b800-a8b2d6f303f6 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:33.914Z InvocationID=b8c71ccc-738f-477d-a25d-6084eb52ad6a ErrorCode=500 owner of resource a514d9bd-7ad5-43e6-a4a1-82801846f755 is jh0003 2021-08-11T16:18:37.368Z [qtp2045766957-14] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "e945dee7-ca2a-418b-b800-a8b2d6f303f6" INVARIANT_UUID = "5fce6f56-b8f4-4441-a8d6-1217f0721dfe" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e945dee7-ca2a-418b-b800-a8b2d6f303f6 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:33.914Z InvocationID=b8c71ccc-738f-477d-a25d-6084eb52ad6a ErrorCode=500 owner of resource a514d9bd-7ad5-43e6-a4a1-82801846f755 is jh0003 2021-08-11T16:18:37.368Z [qtp2045766957-14] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "e945dee7-ca2a-418b-b800-a8b2d6f303f6" INVARIANT_UUID = "5fce6f56-b8f4-4441-a8d6-1217f0721dfe" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e945dee7-ca2a-418b-b800-a8b2d6f303f6 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:33.914Z InvocationID=b8c71ccc-738f-477d-a25d-6084eb52ad6a ErrorCode=500 validate before certification change. resource name=VNF External CP, oldState=NOT_CERTIFIED_CHECKIN, owner userId=jh0003 2021-08-11T16:18:37.368Z [qtp2045766957-14] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "e945dee7-ca2a-418b-b800-a8b2d6f303f6" INVARIANT_UUID = "5fce6f56-b8f4-4441-a8d6-1217f0721dfe" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e945dee7-ca2a-418b-b800-a8b2d6f303f6 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:33.914Z InvocationID=b8c71ccc-738f-477d-a25d-6084eb52ad6a ErrorCode=500 start performing certification change for resource a514d9bd-7ad5-43e6-a4a1-82801846f755 2021-08-11T16:18:37.793Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f013fc5-b582-4922-a9f7-882ef87d78b0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:18:37.793Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f013fc5-b582-4922-a9f7-882ef87d78b0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@50939d1f 2021-08-11T16:18:37.793Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f013fc5-b582-4922-a9f7-882ef87d78b0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:18:37.793Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f013fc5-b582-4922-a9f7-882ef87d78b0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:18:38.002Z [qtp2045766957-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "e945dee7-ca2a-418b-b800-a8b2d6f303f6" INVARIANT_UUID = "5fce6f56-b8f4-4441-a8d6-1217f0721dfe" PREV_VERSION = "" CURR_VERSION = "1.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "cf9121f9-869a-42a4-a3b3-66aff5eeb3a3.assettoscatemplate" CURR_ARTIFACT_UUID = "cf9121f9-869a-42a4-a3b3-66aff5eeb3a3.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VnfExternalCp-template.yml,0,cf9121f9-869a-42a4-a3b3-66aff5eeb3a3.assettoscatemplate,1,f590acd1-c098-45b1-839a-3a973f1181b4" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7f5cfea1-fdbe-451f-9458-c8f543e7063d TargetServiceName=null EntryTimestamp=2021-08-11T16:18:33.914Z InvocationID=b8c71ccc-738f-477d-a25d-6084eb52ad6a ErrorCode=500 Starting to create Csar for component VNF External CP 2021-08-11T16:18:38.321Z [qtp2045766957-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "e945dee7-ca2a-418b-b800-a8b2d6f303f6" INVARIANT_UUID = "5fce6f56-b8f4-4441-a8d6-1217f0721dfe" PREV_VERSION = "" CURR_VERSION = "1.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "cf9121f9-869a-42a4-a3b3-66aff5eeb3a3.assettoscatemplate" CURR_ARTIFACT_UUID = "cf9121f9-869a-42a4-a3b3-66aff5eeb3a3.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VnfExternalCp-template.yml,0,cf9121f9-869a-42a4-a3b3-66aff5eeb3a3.assettoscatemplate,1,f590acd1-c098-45b1-839a-3a973f1181b4" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=7f5cfea1-fdbe-451f-9458-c8f543e7063d TargetServiceName=null EntryTimestamp=2021-08-11T16:18:33.914Z InvocationID=b8c71ccc-738f-477d-a25d-6084eb52ad6a ErrorCode=500 Ended create Csar for component VNF External CP 2021-08-11T16:18:39.087Z [qtp2045766957-14] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "e945dee7-ca2a-418b-b800-a8b2d6f303f6" INVARIANT_UUID = "5fce6f56-b8f4-4441-a8d6-1217f0721dfe" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=cf699a32-1372-43ad-bd43-62f259df4b90 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:33.914Z InvocationID=b8c71ccc-738f-477d-a25d-6084eb52ad6a ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:18:39.088Z [qtp2045766957-14] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "e945dee7-ca2a-418b-b800-a8b2d6f303f6" INVARIANT_UUID = "5fce6f56-b8f4-4441-a8d6-1217f0721dfe" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=cf699a32-1372-43ad-bd43-62f259df4b90 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:33.914Z InvocationID=b8c71ccc-738f-477d-a25d-6084eb52ad6a ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:18:39.088Z [qtp2045766957-14] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "e945dee7-ca2a-418b-b800-a8b2d6f303f6" INVARIANT_UUID = "5fce6f56-b8f4-4441-a8d6-1217f0721dfe" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=cf699a32-1372-43ad-bd43-62f259df4b90 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:33.914Z InvocationID=b8c71ccc-738f-477d-a25d-6084eb52ad6a ErrorCode=500 unlock component a514d9bd-7ad5-43e6-a4a1-82801846f755 2021-08-11T16:18:39.088Z [qtp2045766957-14] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "e945dee7-ca2a-418b-b800-a8b2d6f303f6" INVARIANT_UUID = "5fce6f56-b8f4-4441-a8d6-1217f0721dfe" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=cf699a32-1372-43ad-bd43-62f259df4b90 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:33.914Z InvocationID=b8c71ccc-738f-477d-a25d-6084eb52ad6a ErrorCode=500 During change state, another component a514d9bd-7ad5-43e6-a4a1-82801846f755 has been created/updated 2021-08-11T16:18:39.639Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VNF External CP" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "e945dee7-ca2a-418b-b800-a8b2d6f303f6" INVARIANT_UUID = "5fce6f56-b8f4-4441-a8d6-1217f0721dfe" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5725 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=cf699a32-1372-43ad-bd43-62f259df4b90 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:33.914Z InvocationID=b8c71ccc-738f-477d-a25d-6084eb52ad6a ErrorCode=500 LogTimestamp=2021-08-11T16:18:39.639643Z ResponseDesc=Created Exiting. 2021-08-11T16:18:39.640Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:18:39.654Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=6c8eb8a4-6623-4203-9d91-3b2d46c61970 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:18:39.654465Z InvocationID=94f35b2f-2ab7-4311-997c-fbfe7f972e5d LogTimestamp=2021-08-11T16:18:39.654504Z Entering 2021-08-11T16:18:39.654Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=6c8eb8a4-6623-4203-9d91-3b2d46c61970 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:18:39.654465Z InvocationID=94f35b2f-2ab7-4311-997c-fbfe7f972e5d LogTimestamp=2021-08-11T16:18:39.654504Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:18:39.834Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:18:39.655Z InvocationID=88faef3d-8705-484e-bdf6-56260d8d68bc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id 244ea341-a6db-4cce-8507-7871b5ec5d3d 2021-08-11T16:18:40.440Z [qtp2045766957-16] INFO o.o.s.b.c.l.LifecycleBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:18:39.655Z InvocationID=88faef3d-8705-484e-bdf6-56260d8d68bc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p owner of resource 244ea341-a6db-4cce-8507-7871b5ec5d3d is jh0003 2021-08-11T16:18:40.487Z [qtp2045766957-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "14b7c4c0-9fd2-403b-b42d-462fd67a6a4c" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:18:39.655Z InvocationID=88faef3d-8705-484e-bdf6-56260d8d68bc PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:18:40.487Z [qtp2045766957-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "14b7c4c0-9fd2-403b-b42d-462fd67a6a4c" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2021-08-11T16:18:39.655Z InvocationID=88faef3d-8705-484e-bdf6-56260d8d68bc ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:18:40.700Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf726351-45bc-4bd4-8737-f0fc4ff3fe05 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:18:40.700Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf726351-45bc-4bd4-8737-f0fc4ff3fe05 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@21f55a3d 2021-08-11T16:18:40.700Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf726351-45bc-4bd4-8737-f0fc4ff3fe05 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:18:40.700Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf726351-45bc-4bd4-8737-f0fc4ff3fe05 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:18:41.040Z [qtp2045766957-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "14b7c4c0-9fd2-403b-b42d-462fd67a6a4c" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d4b9d521-a8f2-4a36-a40a-d8331d0c541a TargetServiceName=null EntryTimestamp=2021-08-11T16:18:39.655Z InvocationID=88faef3d-8705-484e-bdf6-56260d8d68bc ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:18:41.040Z [qtp2045766957-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "14b7c4c0-9fd2-403b-b42d-462fd67a6a4c" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d4b9d521-a8f2-4a36-a40a-d8331d0c541a TargetServiceName=null EntryTimestamp=2021-08-11T16:18:39.655Z InvocationID=88faef3d-8705-484e-bdf6-56260d8d68bc ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:18:41.857Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "14b7c4c0-9fd2-403b-b42d-462fd67a6a4c" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VduCp" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d4b9d521-a8f2-4a36-a40a-d8331d0c541a TargetServiceName=null EntryTimestamp=2021-08-11T16:18:39.655Z InvocationID=88faef3d-8705-484e-bdf6-56260d8d68bc ErrorCode=500 lock resource with id ffde7336-c038-416b-82fa-87114fbc8a28 2021-08-11T16:18:42.397Z [qtp2045766957-16] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "14b7c4c0-9fd2-403b-b42d-462fd67a6a4c" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VduCp" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d4b9d521-a8f2-4a36-a40a-d8331d0c541a TargetServiceName=null EntryTimestamp=2021-08-11T16:18:39.655Z InvocationID=88faef3d-8705-484e-bdf6-56260d8d68bc ErrorCode=500 owner of resource ffde7336-c038-416b-82fa-87114fbc8a28 is jh0003 2021-08-11T16:18:43.139Z [qtp2045766957-16] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d4b9d521-a8f2-4a36-a40a-d8331d0c541a" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d4b9d521-a8f2-4a36-a40a-d8331d0c541a TargetServiceName=null EntryTimestamp=2021-08-11T16:18:39.655Z InvocationID=88faef3d-8705-484e-bdf6-56260d8d68bc ErrorCode=500 owner of resource ffde7336-c038-416b-82fa-87114fbc8a28 is jh0003 2021-08-11T16:18:43.139Z [qtp2045766957-16] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d4b9d521-a8f2-4a36-a40a-d8331d0c541a" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d4b9d521-a8f2-4a36-a40a-d8331d0c541a TargetServiceName=null EntryTimestamp=2021-08-11T16:18:39.655Z InvocationID=88faef3d-8705-484e-bdf6-56260d8d68bc ErrorCode=500 validate before certification change. resource name=VDU Cp, oldState=NOT_CERTIFIED_CHECKIN, owner userId=jh0003 2021-08-11T16:18:43.139Z [qtp2045766957-16] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d4b9d521-a8f2-4a36-a40a-d8331d0c541a" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d4b9d521-a8f2-4a36-a40a-d8331d0c541a TargetServiceName=null EntryTimestamp=2021-08-11T16:18:39.655Z InvocationID=88faef3d-8705-484e-bdf6-56260d8d68bc ErrorCode=500 start performing certification change for resource ffde7336-c038-416b-82fa-87114fbc8a28 2021-08-11T16:18:43.472Z [qtp2045766957-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "d4b9d521-a8f2-4a36-a40a-d8331d0c541a" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "" CURR_VERSION = "1.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "244ea341-a6db-4cce-8507-7871b5ec5d3d.assettoscatemplate" CURR_ARTIFACT_UUID = "244ea341-a6db-4cce-8507-7871b5ec5d3d.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduCp-template.yml,0,244ea341-a6db-4cce-8507-7871b5ec5d3d.assettoscatemplate,1,1e51b55c-4013-446c-937c-abcd799aa41f" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b309f831-be7f-4ad1-b936-1a48253dd3cc TargetServiceName=null EntryTimestamp=2021-08-11T16:18:39.655Z InvocationID=88faef3d-8705-484e-bdf6-56260d8d68bc ErrorCode=500 Starting to create Csar for component VDU Cp 2021-08-11T16:18:43.670Z [qtp2045766957-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "d4b9d521-a8f2-4a36-a40a-d8331d0c541a" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "" CURR_VERSION = "1.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "244ea341-a6db-4cce-8507-7871b5ec5d3d.assettoscatemplate" CURR_ARTIFACT_UUID = "244ea341-a6db-4cce-8507-7871b5ec5d3d.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduCp-template.yml,0,244ea341-a6db-4cce-8507-7871b5ec5d3d.assettoscatemplate,1,1e51b55c-4013-446c-937c-abcd799aa41f" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b309f831-be7f-4ad1-b936-1a48253dd3cc TargetServiceName=null EntryTimestamp=2021-08-11T16:18:39.655Z InvocationID=88faef3d-8705-484e-bdf6-56260d8d68bc ErrorCode=500 Ended create Csar for component VDU Cp 2021-08-11T16:18:43.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d0cfa35a-c04b-4788-8350-52a107f017ed PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:18:43.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d0cfa35a-c04b-4788-8350-52a107f017ed PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@63f974f 2021-08-11T16:18:43.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d0cfa35a-c04b-4788-8350-52a107f017ed PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:18:43.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d0cfa35a-c04b-4788-8350-52a107f017ed PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:18:44.539Z [qtp2045766957-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d4b9d521-a8f2-4a36-a40a-d8331d0c541a" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b4705aff-141b-4be7-b4f3-e1de43526044 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:39.655Z InvocationID=88faef3d-8705-484e-bdf6-56260d8d68bc ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:18:44.539Z [qtp2045766957-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d4b9d521-a8f2-4a36-a40a-d8331d0c541a" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b4705aff-141b-4be7-b4f3-e1de43526044 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:39.655Z InvocationID=88faef3d-8705-484e-bdf6-56260d8d68bc ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:18:44.539Z [qtp2045766957-16] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d4b9d521-a8f2-4a36-a40a-d8331d0c541a" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b4705aff-141b-4be7-b4f3-e1de43526044 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:39.655Z InvocationID=88faef3d-8705-484e-bdf6-56260d8d68bc ErrorCode=500 unlock component ffde7336-c038-416b-82fa-87114fbc8a28 2021-08-11T16:18:44.539Z [qtp2045766957-16] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d4b9d521-a8f2-4a36-a40a-d8331d0c541a" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b4705aff-141b-4be7-b4f3-e1de43526044 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:39.655Z InvocationID=88faef3d-8705-484e-bdf6-56260d8d68bc ErrorCode=500 During change state, another component ffde7336-c038-416b-82fa-87114fbc8a28 has been created/updated 2021-08-11T16:18:45.100Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "d4b9d521-a8f2-4a36-a40a-d8331d0c541a" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5445 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b4705aff-141b-4be7-b4f3-e1de43526044 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:39.655Z InvocationID=88faef3d-8705-484e-bdf6-56260d8d68bc ErrorCode=500 LogTimestamp=2021-08-11T16:18:45.100417Z ResponseDesc=Created Exiting. 2021-08-11T16:18:45.101Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:18:45.119Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=3acf4998-e3d2-4b10-a976-6a10972ba3ed InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:18:45.119621Z InvocationID=6c81a743-33d0-4386-8936-69a277b04f92 LogTimestamp=2021-08-11T16:18:45.119686Z Entering 2021-08-11T16:18:45.120Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=3acf4998-e3d2-4b10-a976-6a10972ba3ed InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:18:45.119621Z InvocationID=6c81a743-33d0-4386-8936-69a277b04f92 LogTimestamp=2021-08-11T16:18:45.119686Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:18:45.407Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:18:45.120Z InvocationID=f5661003-ff49-4c78-9f7e-b84840ad168a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:18:45.407Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:18:45.120Z InvocationID=f5661003-ff49-4c78-9f7e-b84840ad168a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:18:45.407Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:18:45.120Z InvocationID=f5661003-ff49-4c78-9f7e-b84840ad168a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:18:45.407Z [qtp2045766957-12] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:18:45.120Z InvocationID=f5661003-ff49-4c78-9f7e-b84840ad168a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:18:45.432Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:18:45.120Z InvocationID=f5661003-ff49-4c78-9f7e-b84840ad168a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name VipCp 2021-08-11T16:18:46.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=74c99017-4137-4dfe-81ff-ff56015d2f2b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:18:46.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=74c99017-4137-4dfe-81ff-ff56015d2f2b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@59a467d8 2021-08-11T16:18:46.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=74c99017-4137-4dfe-81ff-ff56015d2f2b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:18:46.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=74c99017-4137-4dfe-81ff-ff56015d2f2b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:18:47.162Z [qtp2045766957-12] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VIP Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "c1503fb2-a07c-4e4e-81cf-406c5fd4a86a" INVARIANT_UUID = "834fecd0-e9a3-425b-ab78-ceb7f0d08372" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VipCp" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c1503fb2-a07c-4e4e-81cf-406c5fd4a86a EntryTimestamp=2021-08-11T16:18:45.120Z InvocationID=f5661003-ff49-4c78-9f7e-b84840ad168a PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:18:47.163Z [qtp2045766957-12] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VIP Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "c1503fb2-a07c-4e4e-81cf-406c5fd4a86a" INVARIANT_UUID = "834fecd0-e9a3-425b-ab78-ceb7f0d08372" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VipCp" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c1503fb2-a07c-4e4e-81cf-406c5fd4a86a TargetServiceName=null EntryTimestamp=2021-08-11T16:18:45.120Z InvocationID=f5661003-ff49-4c78-9f7e-b84840ad168a ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:18:47.718Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VIP Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "c1503fb2-a07c-4e4e-81cf-406c5fd4a86a" INVARIANT_UUID = "834fecd0-e9a3-425b-ab78-ceb7f0d08372" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VipCp" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c1503fb2-a07c-4e4e-81cf-406c5fd4a86a TargetServiceName=null EntryTimestamp=2021-08-11T16:18:45.120Z InvocationID=f5661003-ff49-4c78-9f7e-b84840ad168a ErrorCode=500 lock resource with id 186eb3fe-431c-49c2-9723-4ba63057ccc6 2021-08-11T16:18:48.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:18:48.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:18:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:18:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:18:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:18:49.653Z [qtp2045766957-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VIP Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "c1503fb2-a07c-4e4e-81cf-406c5fd4a86a" INVARIANT_UUID = "834fecd0-e9a3-425b-ab78-ceb7f0d08372" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "186eb3fe-431c-49c2-9723-4ba63057ccc6.assettoscatemplate" CURR_ARTIFACT_UUID = "186eb3fe-431c-49c2-9723-4ba63057ccc6.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VipCp-template.yml,0,186eb3fe-431c-49c2-9723-4ba63057ccc6.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c40a3d17-a1d3-4394-b192-a20a21297e34 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:45.120Z InvocationID=f5661003-ff49-4c78-9f7e-b84840ad168a ErrorCode=500 Starting to create Csar for component VIP Cp 2021-08-11T16:18:49.835Z [qtp2045766957-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VIP Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "c1503fb2-a07c-4e4e-81cf-406c5fd4a86a" INVARIANT_UUID = "834fecd0-e9a3-425b-ab78-ceb7f0d08372" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "186eb3fe-431c-49c2-9723-4ba63057ccc6.assettoscatemplate" CURR_ARTIFACT_UUID = "186eb3fe-431c-49c2-9723-4ba63057ccc6.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VipCp-template.yml,0,186eb3fe-431c-49c2-9723-4ba63057ccc6.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c40a3d17-a1d3-4394-b192-a20a21297e34 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:45.120Z InvocationID=f5661003-ff49-4c78-9f7e-b84840ad168a ErrorCode=500 Ended create Csar for component VIP Cp 2021-08-11T16:18:49.835Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VIP Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "c1503fb2-a07c-4e4e-81cf-406c5fd4a86a" INVARIANT_UUID = "834fecd0-e9a3-425b-ab78-ceb7f0d08372" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "186eb3fe-431c-49c2-9723-4ba63057ccc6.assettoscatemplate" CURR_ARTIFACT_UUID = "186eb3fe-431c-49c2-9723-4ba63057ccc6.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VipCp-template.yml,0,186eb3fe-431c-49c2-9723-4ba63057ccc6.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c40a3d17-a1d3-4394-b192-a20a21297e34 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:45.120Z InvocationID=f5661003-ff49-4c78-9f7e-b84840ad168a ErrorCode=500 lock resource with id 186eb3fe-431c-49c2-9723-4ba63057ccc6 2021-08-11T16:18:50.546Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=09d003e3-adad-4258-acc4-58af40897330 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:18:50.546Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=09d003e3-adad-4258-acc4-58af40897330 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@15137296 2021-08-11T16:18:50.546Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=09d003e3-adad-4258-acc4-58af40897330 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:18:50.546Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=09d003e3-adad-4258-acc4-58af40897330 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:18:52.357Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VIP Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "c1503fb2-a07c-4e4e-81cf-406c5fd4a86a" INVARIANT_UUID = "834fecd0-e9a3-425b-ab78-ceb7f0d08372" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "186eb3fe-431c-49c2-9723-4ba63057ccc6.assettoscacsar" CURR_ARTIFACT_UUID = "186eb3fe-431c-49c2-9723-4ba63057ccc6.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VipCp-csar.csar,0,186eb3fe-431c-49c2-9723-4ba63057ccc6.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=7237 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=536fe8c4-d33b-4997-ba16-97702b2787f1 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:45.120Z InvocationID=f5661003-ff49-4c78-9f7e-b84840ad168a ErrorCode=500 LogTimestamp=2021-08-11T16:18:52.357343Z ResponseDesc=Created Exiting. 2021-08-11T16:18:52.358Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:18:52.374Z [qtp2045766957-20] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=83dfe378-3459-4934-a5aa-965026fb59c8 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:18:52.373955Z InvocationID=e6d179ea-2cbc-46fc-80d3-8c891f13dad9 LogTimestamp=2021-08-11T16:18:52.373994Z Entering 2021-08-11T16:18:52.374Z [qtp2045766957-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=83dfe378-3459-4934-a5aa-965026fb59c8 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:18:52.373955Z InvocationID=e6d179ea-2cbc-46fc-80d3-8c891f13dad9 LogTimestamp=2021-08-11T16:18:52.373994Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:18:53.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=331129a3-171e-48d3-97be-42f22347217f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:18:53.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=331129a3-171e-48d3-97be-42f22347217f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1ebcf6bb 2021-08-11T16:18:53.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=331129a3-171e-48d3-97be-42f22347217f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:18:53.971Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=331129a3-171e-48d3-97be-42f22347217f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:18:54.754Z [qtp2045766957-20] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:18:52.374Z InvocationID=ed276e9c-fd50-4c31-b7f0-f8d08f9b9f72 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id ad23e7cf-97bc-4368-981d-96424e77a3ee 2021-08-11T16:18:56.266Z [qtp2045766957-20] INFO o.o.s.b.c.l.LifecycleBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:18:52.374Z InvocationID=ed276e9c-fd50-4c31-b7f0-f8d08f9b9f72 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p owner of resource ad23e7cf-97bc-4368-981d-96424e77a3ee is jh0003 2021-08-11T16:18:56.661Z [qtp2045766957-20] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "2eafe324-451d-4cbb-949a-327660833ed6" INVARIANT_UUID = "1a08d182-da17-4352-a1e3-b6174585e43f" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:18:52.374Z InvocationID=ed276e9c-fd50-4c31-b7f0-f8d08f9b9f72 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:18:56.662Z [qtp2045766957-20] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "2eafe324-451d-4cbb-949a-327660833ed6" INVARIANT_UUID = "1a08d182-da17-4352-a1e3-b6174585e43f" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2021-08-11T16:18:52.374Z InvocationID=ed276e9c-fd50-4c31-b7f0-f8d08f9b9f72 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:18:56.850Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=92e08ba7-9541-43dd-815d-102b8b10c3be PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:18:56.850Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=92e08ba7-9541-43dd-815d-102b8b10c3be PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@64439fcc 2021-08-11T16:18:56.850Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=92e08ba7-9541-43dd-815d-102b8b10c3be PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:18:56.850Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=92e08ba7-9541-43dd-815d-102b8b10c3be PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:19:00.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=91edce41-8f0e-4c51-80ac-1c4799f05d6b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:19:00.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=91edce41-8f0e-4c51-80ac-1c4799f05d6b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1ea3ae3b 2021-08-11T16:19:00.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=91edce41-8f0e-4c51-80ac-1c4799f05d6b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:19:00.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=91edce41-8f0e-4c51-80ac-1c4799f05d6b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:19:00.149Z [qtp2045766957-20] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "2eafe324-451d-4cbb-949a-327660833ed6" INVARIANT_UUID = "1a08d182-da17-4352-a1e3-b6174585e43f" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bb675413-ff47-488d-9d7b-fb154e142260 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:52.374Z InvocationID=ed276e9c-fd50-4c31-b7f0-f8d08f9b9f72 ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:19:00.149Z [qtp2045766957-20] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "2eafe324-451d-4cbb-949a-327660833ed6" INVARIANT_UUID = "1a08d182-da17-4352-a1e3-b6174585e43f" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bb675413-ff47-488d-9d7b-fb154e142260 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:52.374Z InvocationID=ed276e9c-fd50-4c31-b7f0-f8d08f9b9f72 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:19:02.164Z [qtp2045766957-20] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "2eafe324-451d-4cbb-949a-327660833ed6" INVARIANT_UUID = "1a08d182-da17-4352-a1e3-b6174585e43f" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualBlockStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bb675413-ff47-488d-9d7b-fb154e142260 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:52.374Z InvocationID=ed276e9c-fd50-4c31-b7f0-f8d08f9b9f72 ErrorCode=500 lock resource with id 255808a9-61df-4b26-801e-c8602d09c06a 2021-08-11T16:19:02.855Z [qtp2045766957-20] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "2eafe324-451d-4cbb-949a-327660833ed6" INVARIANT_UUID = "1a08d182-da17-4352-a1e3-b6174585e43f" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.VirtualBlockStorage" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bb675413-ff47-488d-9d7b-fb154e142260 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:52.374Z InvocationID=ed276e9c-fd50-4c31-b7f0-f8d08f9b9f72 ErrorCode=500 owner of resource 255808a9-61df-4b26-801e-c8602d09c06a is jh0003 2021-08-11T16:19:02.968Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6391d7e6-4561-415a-ab2f-8bc116fd99cf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:19:02.968Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6391d7e6-4561-415a-ab2f-8bc116fd99cf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@102ae8bb 2021-08-11T16:19:02.968Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6391d7e6-4561-415a-ab2f-8bc116fd99cf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:19:02.968Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6391d7e6-4561-415a-ab2f-8bc116fd99cf PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:19:03.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:19:03.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:19:04.258Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:19:05.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed6984ee-1fda-447d-b993-9258a47399e6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:19:05.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed6984ee-1fda-447d-b993-9258a47399e6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@69dacd2b 2021-08-11T16:19:05.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed6984ee-1fda-447d-b993-9258a47399e6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:19:05.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed6984ee-1fda-447d-b993-9258a47399e6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:19:06.082Z [qtp2045766957-20] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "bb675413-ff47-488d-9d7b-fb154e142260" INVARIANT_UUID = "1a08d182-da17-4352-a1e3-b6174585e43f" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bb675413-ff47-488d-9d7b-fb154e142260 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:52.374Z InvocationID=ed276e9c-fd50-4c31-b7f0-f8d08f9b9f72 ErrorCode=500 owner of resource 255808a9-61df-4b26-801e-c8602d09c06a is jh0003 2021-08-11T16:19:06.082Z [qtp2045766957-20] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "bb675413-ff47-488d-9d7b-fb154e142260" INVARIANT_UUID = "1a08d182-da17-4352-a1e3-b6174585e43f" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bb675413-ff47-488d-9d7b-fb154e142260 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:52.374Z InvocationID=ed276e9c-fd50-4c31-b7f0-f8d08f9b9f72 ErrorCode=500 validate before certification change. resource name=VDU VirtualBlockStorage, oldState=NOT_CERTIFIED_CHECKIN, owner userId=jh0003 2021-08-11T16:19:06.082Z [qtp2045766957-20] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "bb675413-ff47-488d-9d7b-fb154e142260" INVARIANT_UUID = "1a08d182-da17-4352-a1e3-b6174585e43f" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bb675413-ff47-488d-9d7b-fb154e142260 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:52.374Z InvocationID=ed276e9c-fd50-4c31-b7f0-f8d08f9b9f72 ErrorCode=500 start performing certification change for resource 255808a9-61df-4b26-801e-c8602d09c06a 2021-08-11T16:19:07.163Z [qtp2045766957-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "bb675413-ff47-488d-9d7b-fb154e142260" INVARIANT_UUID = "1a08d182-da17-4352-a1e3-b6174585e43f" PREV_VERSION = "" CURR_VERSION = "1.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "ad23e7cf-97bc-4368-981d-96424e77a3ee.assettoscatemplate" CURR_ARTIFACT_UUID = "ad23e7cf-97bc-4368-981d-96424e77a3ee.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualblockstorage-template.yml,0,ad23e7cf-97bc-4368-981d-96424e77a3ee.assettoscatemplate,1,32bed20e-6e60-4ab2-9d90-05081d5cdbbd" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=69e6a104-f039-4b24-94b0-fd6a52e25aec TargetServiceName=null EntryTimestamp=2021-08-11T16:18:52.374Z InvocationID=ed276e9c-fd50-4c31-b7f0-f8d08f9b9f72 ErrorCode=500 Starting to create Csar for component VDU VirtualBlockStorage 2021-08-11T16:19:07.345Z [qtp2045766957-20] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "bb675413-ff47-488d-9d7b-fb154e142260" INVARIANT_UUID = "1a08d182-da17-4352-a1e3-b6174585e43f" PREV_VERSION = "" CURR_VERSION = "1.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "ad23e7cf-97bc-4368-981d-96424e77a3ee.assettoscatemplate" CURR_ARTIFACT_UUID = "ad23e7cf-97bc-4368-981d-96424e77a3ee.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduVirtualblockstorage-template.yml,0,ad23e7cf-97bc-4368-981d-96424e77a3ee.assettoscatemplate,1,32bed20e-6e60-4ab2-9d90-05081d5cdbbd" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=69e6a104-f039-4b24-94b0-fd6a52e25aec TargetServiceName=null EntryTimestamp=2021-08-11T16:18:52.374Z InvocationID=ed276e9c-fd50-4c31-b7f0-f8d08f9b9f72 ErrorCode=500 Ended create Csar for component VDU VirtualBlockStorage 2021-08-11T16:19:08.283Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b4b6fbf-bbb2-4078-9b13-fb53e22e073a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:19:08.283Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b4b6fbf-bbb2-4078-9b13-fb53e22e073a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@111f9223 2021-08-11T16:19:08.284Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b4b6fbf-bbb2-4078-9b13-fb53e22e073a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:19:08.284Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b4b6fbf-bbb2-4078-9b13-fb53e22e073a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:19:08.977Z [qtp2045766957-20] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "bb675413-ff47-488d-9d7b-fb154e142260" INVARIANT_UUID = "1a08d182-da17-4352-a1e3-b6174585e43f" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3bced5fc-0874-4270-abdf-e3ca05e289f8 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:52.374Z InvocationID=ed276e9c-fd50-4c31-b7f0-f8d08f9b9f72 ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:19:08.977Z [qtp2045766957-20] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "bb675413-ff47-488d-9d7b-fb154e142260" INVARIANT_UUID = "1a08d182-da17-4352-a1e3-b6174585e43f" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3bced5fc-0874-4270-abdf-e3ca05e289f8 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:52.374Z InvocationID=ed276e9c-fd50-4c31-b7f0-f8d08f9b9f72 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:19:08.977Z [qtp2045766957-20] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "bb675413-ff47-488d-9d7b-fb154e142260" INVARIANT_UUID = "1a08d182-da17-4352-a1e3-b6174585e43f" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3bced5fc-0874-4270-abdf-e3ca05e289f8 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:52.374Z InvocationID=ed276e9c-fd50-4c31-b7f0-f8d08f9b9f72 ErrorCode=500 unlock component 255808a9-61df-4b26-801e-c8602d09c06a 2021-08-11T16:19:08.977Z [qtp2045766957-20] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "bb675413-ff47-488d-9d7b-fb154e142260" INVARIANT_UUID = "1a08d182-da17-4352-a1e3-b6174585e43f" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3bced5fc-0874-4270-abdf-e3ca05e289f8 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:52.374Z InvocationID=ed276e9c-fd50-4c31-b7f0-f8d08f9b9f72 ErrorCode=500 During change state, another component 255808a9-61df-4b26-801e-c8602d09c06a has been created/updated 2021-08-11T16:19:09.530Z [qtp2045766957-20] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU VirtualBlockStorage" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "bb675413-ff47-488d-9d7b-fb154e142260" INVARIANT_UUID = "1a08d182-da17-4352-a1e3-b6174585e43f" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=17156 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=3bced5fc-0874-4270-abdf-e3ca05e289f8 TargetServiceName=null EntryTimestamp=2021-08-11T16:18:52.374Z InvocationID=ed276e9c-fd50-4c31-b7f0-f8d08f9b9f72 ErrorCode=500 LogTimestamp=2021-08-11T16:19:09.530498Z ResponseDesc=Created Exiting. 2021-08-11T16:19:09.531Z [qtp2045766957-20] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:19:09.550Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=47ea054a-44eb-49ec-8976-816c1eb509d6 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:19:09.550589Z InvocationID=71d8c381-a6b4-4f14-a53a-e0ec6cb21efc LogTimestamp=2021-08-11T16:19:09.550632Z Entering 2021-08-11T16:19:09.551Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=47ea054a-44eb-49ec-8976-816c1eb509d6 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:19:09.550589Z InvocationID=71d8c381-a6b4-4f14-a53a-e0ec6cb21efc LogTimestamp=2021-08-11T16:19:09.550632Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:19:09.802Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:19:09.551Z InvocationID=6b94029f-67f5-4255-a925-cb915141f76f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id 4e1b14a8-767a-4a5d-893c-96c9807d8423 2021-08-11T16:19:10.408Z [qtp2045766957-12] INFO o.o.s.b.c.l.LifecycleBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:19:09.551Z InvocationID=6b94029f-67f5-4255-a925-cb915141f76f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p owner of resource 4e1b14a8-767a-4a5d-893c-96c9807d8423 is jh0003 2021-08-11T16:19:10.463Z [qtp2045766957-12] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "2a750ba8-ef84-4ae6-b28c-227391454a37" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:19:09.551Z InvocationID=6b94029f-67f5-4255-a925-cb915141f76f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:19:10.464Z [qtp2045766957-12] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "2a750ba8-ef84-4ae6-b28c-227391454a37" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2021-08-11T16:19:09.551Z InvocationID=6b94029f-67f5-4255-a925-cb915141f76f ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:19:10.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8ae4db33-8f9c-4c67-b645-e91dfbbb43b9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:19:10.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8ae4db33-8f9c-4c67-b645-e91dfbbb43b9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@12ce244d 2021-08-11T16:19:10.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8ae4db33-8f9c-4c67-b645-e91dfbbb43b9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:19:10.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8ae4db33-8f9c-4c67-b645-e91dfbbb43b9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:19:11.095Z [qtp2045766957-12] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "2a750ba8-ef84-4ae6-b28c-227391454a37" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c4d88e2d-c692-4ad3-8297-41c1cd2a73ac TargetServiceName=null EntryTimestamp=2021-08-11T16:19:09.551Z InvocationID=6b94029f-67f5-4255-a925-cb915141f76f ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:19:11.096Z [qtp2045766957-12] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "2a750ba8-ef84-4ae6-b28c-227391454a37" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c4d88e2d-c692-4ad3-8297-41c1cd2a73ac TargetServiceName=null EntryTimestamp=2021-08-11T16:19:09.551Z InvocationID=6b94029f-67f5-4255-a925-cb915141f76f ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:19:11.855Z [qtp2045766957-12] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "2a750ba8-ef84-4ae6-b28c-227391454a37" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VNF" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c4d88e2d-c692-4ad3-8297-41c1cd2a73ac TargetServiceName=null EntryTimestamp=2021-08-11T16:19:09.551Z InvocationID=6b94029f-67f5-4255-a925-cb915141f76f ErrorCode=500 lock resource with id 6f4c3b63-6deb-4c83-934a-a74273b1ec8f 2021-08-11T16:19:12.383Z [qtp2045766957-12] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "2a750ba8-ef84-4ae6-b28c-227391454a37" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VNF" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c4d88e2d-c692-4ad3-8297-41c1cd2a73ac TargetServiceName=null EntryTimestamp=2021-08-11T16:19:09.551Z InvocationID=6b94029f-67f5-4255-a925-cb915141f76f ErrorCode=500 owner of resource 6f4c3b63-6deb-4c83-934a-a74273b1ec8f is jh0003 2021-08-11T16:19:13.306Z [qtp2045766957-12] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "c4d88e2d-c692-4ad3-8297-41c1cd2a73ac" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c4d88e2d-c692-4ad3-8297-41c1cd2a73ac TargetServiceName=null EntryTimestamp=2021-08-11T16:19:09.551Z InvocationID=6b94029f-67f5-4255-a925-cb915141f76f ErrorCode=500 owner of resource 6f4c3b63-6deb-4c83-934a-a74273b1ec8f is jh0003 2021-08-11T16:19:13.306Z [qtp2045766957-12] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "c4d88e2d-c692-4ad3-8297-41c1cd2a73ac" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c4d88e2d-c692-4ad3-8297-41c1cd2a73ac TargetServiceName=null EntryTimestamp=2021-08-11T16:19:09.551Z InvocationID=6b94029f-67f5-4255-a925-cb915141f76f ErrorCode=500 validate before certification change. resource name=VNF, oldState=NOT_CERTIFIED_CHECKIN, owner userId=jh0003 2021-08-11T16:19:13.306Z [qtp2045766957-12] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "c4d88e2d-c692-4ad3-8297-41c1cd2a73ac" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=c4d88e2d-c692-4ad3-8297-41c1cd2a73ac TargetServiceName=null EntryTimestamp=2021-08-11T16:19:09.551Z InvocationID=6b94029f-67f5-4255-a925-cb915141f76f ErrorCode=500 start performing certification change for resource 6f4c3b63-6deb-4c83-934a-a74273b1ec8f 2021-08-11T16:19:13.764Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d7e8d6be-3646-463f-b1b1-0b71ef66a99e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:19:13.764Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d7e8d6be-3646-463f-b1b1-0b71ef66a99e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@55e6699f 2021-08-11T16:19:13.764Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d7e8d6be-3646-463f-b1b1-0b71ef66a99e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:19:13.764Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d7e8d6be-3646-463f-b1b1-0b71ef66a99e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:19:13.937Z [qtp2045766957-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "c4d88e2d-c692-4ad3-8297-41c1cd2a73ac" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "" CURR_VERSION = "1.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "4e1b14a8-767a-4a5d-893c-96c9807d8423.assettoscatemplate" CURR_ARTIFACT_UUID = "4e1b14a8-767a-4a5d-893c-96c9807d8423.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vnf-template.yml,0,4e1b14a8-767a-4a5d-893c-96c9807d8423.assettoscatemplate,1,ee2d242e-9664-4f75-808d-f0199cda448f" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=acedf2af-83f9-436e-ad53-ac4325c15af7 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:09.551Z InvocationID=6b94029f-67f5-4255-a925-cb915141f76f ErrorCode=500 Starting to create Csar for component VNF 2021-08-11T16:19:14.113Z [qtp2045766957-12] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "c4d88e2d-c692-4ad3-8297-41c1cd2a73ac" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "" CURR_VERSION = "1.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "4e1b14a8-767a-4a5d-893c-96c9807d8423.assettoscatemplate" CURR_ARTIFACT_UUID = "4e1b14a8-767a-4a5d-893c-96c9807d8423.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vnf-template.yml,0,4e1b14a8-767a-4a5d-893c-96c9807d8423.assettoscatemplate,1,ee2d242e-9664-4f75-808d-f0199cda448f" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=acedf2af-83f9-436e-ad53-ac4325c15af7 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:09.551Z InvocationID=6b94029f-67f5-4255-a925-cb915141f76f ErrorCode=500 Ended create Csar for component VNF 2021-08-11T16:19:15.150Z [qtp2045766957-12] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "c4d88e2d-c692-4ad3-8297-41c1cd2a73ac" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=fe185b77-f936-41f1-b1b2-2f2c1aaf0ab7 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:09.551Z InvocationID=6b94029f-67f5-4255-a925-cb915141f76f ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:19:15.151Z [qtp2045766957-12] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "c4d88e2d-c692-4ad3-8297-41c1cd2a73ac" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=fe185b77-f936-41f1-b1b2-2f2c1aaf0ab7 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:09.551Z InvocationID=6b94029f-67f5-4255-a925-cb915141f76f ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:19:15.151Z [qtp2045766957-12] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "c4d88e2d-c692-4ad3-8297-41c1cd2a73ac" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=fe185b77-f936-41f1-b1b2-2f2c1aaf0ab7 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:09.551Z InvocationID=6b94029f-67f5-4255-a925-cb915141f76f ErrorCode=500 unlock component 6f4c3b63-6deb-4c83-934a-a74273b1ec8f 2021-08-11T16:19:15.151Z [qtp2045766957-12] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "c4d88e2d-c692-4ad3-8297-41c1cd2a73ac" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=fe185b77-f936-41f1-b1b2-2f2c1aaf0ab7 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:09.551Z InvocationID=6b94029f-67f5-4255-a925-cb915141f76f ErrorCode=500 During change state, another component 6f4c3b63-6deb-4c83-934a-a74273b1ec8f has been created/updated 2021-08-11T16:19:15.870Z [qtp2045766957-12] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "c4d88e2d-c692-4ad3-8297-41c1cd2a73ac" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=6318 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=fe185b77-f936-41f1-b1b2-2f2c1aaf0ab7 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:09.551Z InvocationID=6b94029f-67f5-4255-a925-cb915141f76f ErrorCode=500 LogTimestamp=2021-08-11T16:19:15.869919Z ResponseDesc=Created Exiting. 2021-08-11T16:19:15.870Z [qtp2045766957-12] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:19:15.888Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=b6d3406f-d2ea-4397-9bc6-4fd601863d7e InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:19:15.887774Z InvocationID=127cc47c-9526-4dbf-9683-352ef3a008a8 LogTimestamp=2021-08-11T16:19:15.887814Z Entering 2021-08-11T16:19:15.888Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=b6d3406f-d2ea-4397-9bc6-4fd601863d7e InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:19:15.887774Z InvocationID=127cc47c-9526-4dbf-9683-352ef3a008a8 LogTimestamp=2021-08-11T16:19:15.887814Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:19:16.284Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:19:15.888Z InvocationID=9a98ca9c-83ea-408b-b53f-bfc7d89d7550 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id c92410eb-97df-4378-9616-a7005a777f41 2021-08-11T16:19:17.266Z [qtp2045766957-16] INFO o.o.s.b.c.l.LifecycleBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:19:15.888Z InvocationID=9a98ca9c-83ea-408b-b53f-bfc7d89d7550 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p owner of resource c92410eb-97df-4378-9616-a7005a777f41 is jh0003 2021-08-11T16:19:17.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=395b4cc3-1948-4198-93b5-745fa2dc64dc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:19:17.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=395b4cc3-1948-4198-93b5-745fa2dc64dc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6a6a559d 2021-08-11T16:19:17.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=395b4cc3-1948-4198-93b5-745fa2dc64dc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:19:17.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=395b4cc3-1948-4198-93b5-745fa2dc64dc PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:19:17.475Z [qtp2045766957-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "NS" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "92a5acf8-77cd-493c-959c-253b4551ea92" INVARIANT_UUID = "cc9a00a8-135c-4bbb-a261-ed8f9c0bba70" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:19:15.888Z InvocationID=9a98ca9c-83ea-408b-b53f-bfc7d89d7550 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:19:17.476Z [qtp2045766957-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "NS" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "92a5acf8-77cd-493c-959c-253b4551ea92" INVARIANT_UUID = "cc9a00a8-135c-4bbb-a261-ed8f9c0bba70" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2021-08-11T16:19:15.888Z InvocationID=9a98ca9c-83ea-408b-b53f-bfc7d89d7550 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:19:18.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:19:18.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:19:18.464Z [qtp2045766957-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "NS" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "92a5acf8-77cd-493c-959c-253b4551ea92" INVARIANT_UUID = "cc9a00a8-135c-4bbb-a261-ed8f9c0bba70" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=dee90386-2067-41de-9ff6-c1c0abdb7b4e TargetServiceName=null EntryTimestamp=2021-08-11T16:19:15.888Z InvocationID=9a98ca9c-83ea-408b-b53f-bfc7d89d7550 ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:19:18.464Z [qtp2045766957-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "NS" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "92a5acf8-77cd-493c-959c-253b4551ea92" INVARIANT_UUID = "cc9a00a8-135c-4bbb-a261-ed8f9c0bba70" PREV_VERSION = "1.0" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=dee90386-2067-41de-9ff6-c1c0abdb7b4e TargetServiceName=null EntryTimestamp=2021-08-11T16:19:15.888Z InvocationID=9a98ca9c-83ea-408b-b53f-bfc7d89d7550 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:19:19.197Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "92a5acf8-77cd-493c-959c-253b4551ea92" INVARIANT_UUID = "cc9a00a8-135c-4bbb-a261-ed8f9c0bba70" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NS" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=dee90386-2067-41de-9ff6-c1c0abdb7b4e TargetServiceName=null EntryTimestamp=2021-08-11T16:19:15.888Z InvocationID=9a98ca9c-83ea-408b-b53f-bfc7d89d7550 ErrorCode=500 lock resource with id 366b932a-f324-4f2a-b8fd-876ae7c13bf3 2021-08-11T16:19:19.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd47cc26-9be8-41e7-905f-55ae1afaea26 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:19:19.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd47cc26-9be8-41e7-905f-55ae1afaea26 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7894c79 2021-08-11T16:19:19.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd47cc26-9be8-41e7-905f-55ae1afaea26 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:19:19.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd47cc26-9be8-41e7-905f-55ae1afaea26 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:19:19.782Z [qtp2045766957-16] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "NS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "92a5acf8-77cd-493c-959c-253b4551ea92" INVARIANT_UUID = "cc9a00a8-135c-4bbb-a261-ed8f9c0bba70" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.NS" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=dee90386-2067-41de-9ff6-c1c0abdb7b4e TargetServiceName=null EntryTimestamp=2021-08-11T16:19:15.888Z InvocationID=9a98ca9c-83ea-408b-b53f-bfc7d89d7550 ErrorCode=500 owner of resource 366b932a-f324-4f2a-b8fd-876ae7c13bf3 is jh0003 2021-08-11T16:19:20.652Z [qtp2045766957-16] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "NS" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "dee90386-2067-41de-9ff6-c1c0abdb7b4e" INVARIANT_UUID = "cc9a00a8-135c-4bbb-a261-ed8f9c0bba70" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=dee90386-2067-41de-9ff6-c1c0abdb7b4e TargetServiceName=null EntryTimestamp=2021-08-11T16:19:15.888Z InvocationID=9a98ca9c-83ea-408b-b53f-bfc7d89d7550 ErrorCode=500 owner of resource 366b932a-f324-4f2a-b8fd-876ae7c13bf3 is jh0003 2021-08-11T16:19:20.652Z [qtp2045766957-16] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "NS" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "dee90386-2067-41de-9ff6-c1c0abdb7b4e" INVARIANT_UUID = "cc9a00a8-135c-4bbb-a261-ed8f9c0bba70" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=dee90386-2067-41de-9ff6-c1c0abdb7b4e TargetServiceName=null EntryTimestamp=2021-08-11T16:19:15.888Z InvocationID=9a98ca9c-83ea-408b-b53f-bfc7d89d7550 ErrorCode=500 validate before certification change. resource name=NS, oldState=NOT_CERTIFIED_CHECKIN, owner userId=jh0003 2021-08-11T16:19:20.652Z [qtp2045766957-16] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "NS" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "dee90386-2067-41de-9ff6-c1c0abdb7b4e" INVARIANT_UUID = "cc9a00a8-135c-4bbb-a261-ed8f9c0bba70" PREV_VERSION = "1.1" CURR_VERSION = "1.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=dee90386-2067-41de-9ff6-c1c0abdb7b4e TargetServiceName=null EntryTimestamp=2021-08-11T16:19:15.888Z InvocationID=9a98ca9c-83ea-408b-b53f-bfc7d89d7550 ErrorCode=500 start performing certification change for resource 366b932a-f324-4f2a-b8fd-876ae7c13bf3 2021-08-11T16:19:20.946Z [qtp2045766957-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "dee90386-2067-41de-9ff6-c1c0abdb7b4e" INVARIANT_UUID = "cc9a00a8-135c-4bbb-a261-ed8f9c0bba70" PREV_VERSION = "" CURR_VERSION = "1.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "c92410eb-97df-4378-9616-a7005a777f41.assettoscatemplate" CURR_ARTIFACT_UUID = "c92410eb-97df-4378-9616-a7005a777f41.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Ns-template.yml,0,c92410eb-97df-4378-9616-a7005a777f41.assettoscatemplate,1,d0a3cf09-fa21-41b7-b815-0a29685b06ac" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=6b6723eb-95e2-4f83-9025-c157bcf492e0 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:15.888Z InvocationID=9a98ca9c-83ea-408b-b53f-bfc7d89d7550 ErrorCode=500 Starting to create Csar for component NS 2021-08-11T16:19:21.112Z [qtp2045766957-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "NS" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "dee90386-2067-41de-9ff6-c1c0abdb7b4e" INVARIANT_UUID = "cc9a00a8-135c-4bbb-a261-ed8f9c0bba70" PREV_VERSION = "" CURR_VERSION = "1.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "c92410eb-97df-4378-9616-a7005a777f41.assettoscatemplate" CURR_ARTIFACT_UUID = "c92410eb-97df-4378-9616-a7005a777f41.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Ns-template.yml,0,c92410eb-97df-4378-9616-a7005a777f41.assettoscatemplate,1,d0a3cf09-fa21-41b7-b815-0a29685b06ac" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=6b6723eb-95e2-4f83-9025-c157bcf492e0 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:15.888Z InvocationID=9a98ca9c-83ea-408b-b53f-bfc7d89d7550 ErrorCode=500 Ended create Csar for component NS 2021-08-11T16:19:21.918Z [qtp2045766957-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "NS" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "dee90386-2067-41de-9ff6-c1c0abdb7b4e" INVARIANT_UUID = "cc9a00a8-135c-4bbb-a261-ed8f9c0bba70" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d3a4bf39-2628-40fd-a19a-9f33d6d01a24 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:15.888Z InvocationID=9a98ca9c-83ea-408b-b53f-bfc7d89d7550 ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:19:21.918Z [qtp2045766957-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "NS" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "dee90386-2067-41de-9ff6-c1c0abdb7b4e" INVARIANT_UUID = "cc9a00a8-135c-4bbb-a261-ed8f9c0bba70" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d3a4bf39-2628-40fd-a19a-9f33d6d01a24 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:15.888Z InvocationID=9a98ca9c-83ea-408b-b53f-bfc7d89d7550 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:19:21.918Z [qtp2045766957-16] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "NS" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "dee90386-2067-41de-9ff6-c1c0abdb7b4e" INVARIANT_UUID = "cc9a00a8-135c-4bbb-a261-ed8f9c0bba70" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d3a4bf39-2628-40fd-a19a-9f33d6d01a24 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:15.888Z InvocationID=9a98ca9c-83ea-408b-b53f-bfc7d89d7550 ErrorCode=500 unlock component 366b932a-f324-4f2a-b8fd-876ae7c13bf3 2021-08-11T16:19:21.918Z [qtp2045766957-16] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "NS" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "dee90386-2067-41de-9ff6-c1c0abdb7b4e" INVARIANT_UUID = "cc9a00a8-135c-4bbb-a261-ed8f9c0bba70" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d3a4bf39-2628-40fd-a19a-9f33d6d01a24 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:15.888Z InvocationID=9a98ca9c-83ea-408b-b53f-bfc7d89d7550 ErrorCode=500 During change state, another component 366b932a-f324-4f2a-b8fd-876ae7c13bf3 has been created/updated 2021-08-11T16:19:22.474Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "NS" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "dee90386-2067-41de-9ff6-c1c0abdb7b4e" INVARIANT_UUID = "cc9a00a8-135c-4bbb-a261-ed8f9c0bba70" PREV_VERSION = "1.1" CURR_VERSION = "2.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=6586 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=d3a4bf39-2628-40fd-a19a-9f33d6d01a24 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:15.888Z InvocationID=9a98ca9c-83ea-408b-b53f-bfc7d89d7550 ErrorCode=500 LogTimestamp=2021-08-11T16:19:22.474051Z ResponseDesc=Created Exiting. 2021-08-11T16:19:22.474Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:19:22.486Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=e556a81b-0eb5-46b9-956b-8ab6ba7bab7c InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:19:22.486138Z InvocationID=eb2e280d-9132-4839-952b-267835d5444b LogTimestamp=2021-08-11T16:19:22.486175Z Entering 2021-08-11T16:19:22.486Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=e556a81b-0eb5-46b9-956b-8ab6ba7bab7c InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:19:22.486138Z InvocationID=eb2e280d-9132-4839-952b-267835d5444b LogTimestamp=2021-08-11T16:19:22.486175Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:19:22.711Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb2739c9-ee75-4c04-b257-dd83b532c889 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:19:22.711Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb2739c9-ee75-4c04-b257-dd83b532c889 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6486fc89 2021-08-11T16:19:22.711Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb2739c9-ee75-4c04-b257-dd83b532c889 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:19:22.711Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb2739c9-ee75-4c04-b257-dd83b532c889 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:19:23.069Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:19:22.486Z InvocationID=af79847c-ec47-472e-8dac-aab35d43e011 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id e2b01fff-6dd1-4744-9902-4741b03fe8df 2021-08-11T16:19:23.696Z [qtp2045766957-14] INFO o.o.s.b.c.l.LifecycleBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:19:22.486Z InvocationID=af79847c-ec47-472e-8dac-aab35d43e011 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p owner of resource e2b01fff-6dd1-4744-9902-4741b03fe8df is jh0003 2021-08-11T16:19:23.739Z [qtp2045766957-14] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "2b42282c-cc1a-4540-95ed-7af4d76d7cc0" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "2.0" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:19:22.486Z InvocationID=af79847c-ec47-472e-8dac-aab35d43e011 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:19:23.739Z [qtp2045766957-14] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "2b42282c-cc1a-4540-95ed-7af4d76d7cc0" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "2.0" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2021-08-11T16:19:22.486Z InvocationID=af79847c-ec47-472e-8dac-aab35d43e011 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:19:24.014Z [qtp2045766957-14] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "2b42282c-cc1a-4540-95ed-7af4d76d7cc0" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "2.0" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bb6cb0d8-761b-4d09-8d4d-3c0e11ac91d6 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:22.486Z InvocationID=af79847c-ec47-472e-8dac-aab35d43e011 ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:19:24.015Z [qtp2045766957-14] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "2b42282c-cc1a-4540-95ed-7af4d76d7cc0" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "2.0" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bb6cb0d8-761b-4d09-8d4d-3c0e11ac91d6 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:22.486Z InvocationID=af79847c-ec47-472e-8dac-aab35d43e011 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:19:24.786Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "2b42282c-cc1a-4540-95ed-7af4d76d7cc0" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "2.1" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VNF" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bb6cb0d8-761b-4d09-8d4d-3c0e11ac91d6 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:22.486Z InvocationID=af79847c-ec47-472e-8dac-aab35d43e011 ErrorCode=500 lock resource with id 0a3b2f9f-4487-4fa8-91c9-76ea30f1193d 2021-08-11T16:19:25.317Z [qtp2045766957-14] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "2b42282c-cc1a-4540-95ed-7af4d76d7cc0" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "2.1" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VNF" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bb6cb0d8-761b-4d09-8d4d-3c0e11ac91d6 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:22.486Z InvocationID=af79847c-ec47-472e-8dac-aab35d43e011 ErrorCode=500 owner of resource 0a3b2f9f-4487-4fa8-91c9-76ea30f1193d is jh0003 2021-08-11T16:19:25.953Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=280401a7-28ee-4b88-800f-d3c25ebd1624 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:19:25.953Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=280401a7-28ee-4b88-800f-d3c25ebd1624 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2cacb17 2021-08-11T16:19:25.953Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=280401a7-28ee-4b88-800f-d3c25ebd1624 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:19:25.953Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=280401a7-28ee-4b88-800f-d3c25ebd1624 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:19:27.385Z [qtp2045766957-14] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "bb6cb0d8-761b-4d09-8d4d-3c0e11ac91d6" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "2.1" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bb6cb0d8-761b-4d09-8d4d-3c0e11ac91d6 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:22.486Z InvocationID=af79847c-ec47-472e-8dac-aab35d43e011 ErrorCode=500 owner of resource 0a3b2f9f-4487-4fa8-91c9-76ea30f1193d is jh0003 2021-08-11T16:19:27.385Z [qtp2045766957-14] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "bb6cb0d8-761b-4d09-8d4d-3c0e11ac91d6" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "2.1" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bb6cb0d8-761b-4d09-8d4d-3c0e11ac91d6 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:22.486Z InvocationID=af79847c-ec47-472e-8dac-aab35d43e011 ErrorCode=500 validate before certification change. resource name=VNF, oldState=NOT_CERTIFIED_CHECKIN, owner userId=jh0003 2021-08-11T16:19:27.385Z [qtp2045766957-14] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "bb6cb0d8-761b-4d09-8d4d-3c0e11ac91d6" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "2.1" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=bb6cb0d8-761b-4d09-8d4d-3c0e11ac91d6 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:22.486Z InvocationID=af79847c-ec47-472e-8dac-aab35d43e011 ErrorCode=500 start performing certification change for resource 0a3b2f9f-4487-4fa8-91c9-76ea30f1193d 2021-08-11T16:19:28.850Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ed83909-19f8-49a0-93f9-2a8103610302 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:19:28.850Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ed83909-19f8-49a0-93f9-2a8103610302 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3ab4a06e 2021-08-11T16:19:28.850Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ed83909-19f8-49a0-93f9-2a8103610302 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:19:28.851Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ed83909-19f8-49a0-93f9-2a8103610302 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:19:28.860Z [qtp2045766957-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "bb6cb0d8-761b-4d09-8d4d-3c0e11ac91d6" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "" CURR_VERSION = "2.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "6f4c3b63-6deb-4c83-934a-a74273b1ec8f.assettoscatemplate" CURR_ARTIFACT_UUID = "6f4c3b63-6deb-4c83-934a-a74273b1ec8f.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vnf-template.yml,0,6f4c3b63-6deb-4c83-934a-a74273b1ec8f.assettoscatemplate,2,acedf2af-83f9-436e-ad53-ac4325c15af7" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=fc916c11-b15c-493b-b347-9577bc374e88 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:22.486Z InvocationID=af79847c-ec47-472e-8dac-aab35d43e011 ErrorCode=500 Starting to create Csar for component VNF 2021-08-11T16:19:29.013Z [qtp2045766957-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "bb6cb0d8-761b-4d09-8d4d-3c0e11ac91d6" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "" CURR_VERSION = "2.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "6f4c3b63-6deb-4c83-934a-a74273b1ec8f.assettoscatemplate" CURR_ARTIFACT_UUID = "6f4c3b63-6deb-4c83-934a-a74273b1ec8f.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Vnf-template.yml,0,6f4c3b63-6deb-4c83-934a-a74273b1ec8f.assettoscatemplate,2,acedf2af-83f9-436e-ad53-ac4325c15af7" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=fc916c11-b15c-493b-b347-9577bc374e88 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:22.486Z InvocationID=af79847c-ec47-472e-8dac-aab35d43e011 ErrorCode=500 Ended create Csar for component VNF 2021-08-11T16:19:30.498Z [qtp2045766957-14] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "bb6cb0d8-761b-4d09-8d4d-3c0e11ac91d6" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "2.1" CURR_VERSION = "3.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e4b3ee87-f22d-4a29-be59-5247fb6ea4d7 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:22.486Z InvocationID=af79847c-ec47-472e-8dac-aab35d43e011 ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:19:30.498Z [qtp2045766957-14] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "bb6cb0d8-761b-4d09-8d4d-3c0e11ac91d6" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "2.1" CURR_VERSION = "3.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e4b3ee87-f22d-4a29-be59-5247fb6ea4d7 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:22.486Z InvocationID=af79847c-ec47-472e-8dac-aab35d43e011 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:19:30.498Z [qtp2045766957-14] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "bb6cb0d8-761b-4d09-8d4d-3c0e11ac91d6" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "2.1" CURR_VERSION = "3.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e4b3ee87-f22d-4a29-be59-5247fb6ea4d7 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:22.486Z InvocationID=af79847c-ec47-472e-8dac-aab35d43e011 ErrorCode=500 unlock component 0a3b2f9f-4487-4fa8-91c9-76ea30f1193d 2021-08-11T16:19:30.498Z [qtp2045766957-14] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "bb6cb0d8-761b-4d09-8d4d-3c0e11ac91d6" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "2.1" CURR_VERSION = "3.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e4b3ee87-f22d-4a29-be59-5247fb6ea4d7 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:22.486Z InvocationID=af79847c-ec47-472e-8dac-aab35d43e011 ErrorCode=500 During change state, another component 0a3b2f9f-4487-4fa8-91c9-76ea30f1193d has been created/updated 2021-08-11T16:19:31.049Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VNF" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "bb6cb0d8-761b-4d09-8d4d-3c0e11ac91d6" INVARIANT_UUID = "f1cc0aa0-7242-4a48-aad1-9198115decba" PREV_VERSION = "2.1" CURR_VERSION = "3.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=8562 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=e4b3ee87-f22d-4a29-be59-5247fb6ea4d7 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:22.486Z InvocationID=af79847c-ec47-472e-8dac-aab35d43e011 ErrorCode=500 LogTimestamp=2021-08-11T16:19:31.048826Z ResponseDesc=Created Exiting. 2021-08-11T16:19:31.049Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:19:31.061Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=4eb9acf4-0408-4632-bc2d-35445aefd70e InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:19:31.061110Z InvocationID=a8f4c7df-6ed2-4694-836b-f6557db3fe6e LogTimestamp=2021-08-11T16:19:31.061161Z Entering 2021-08-11T16:19:31.061Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=4eb9acf4-0408-4632-bc2d-35445aefd70e InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:19:31.061110Z InvocationID=a8f4c7df-6ed2-4694-836b-f6557db3fe6e LogTimestamp=2021-08-11T16:19:31.061161Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:19:31.568Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:19:31.061Z InvocationID=4bfb78fa-8573-4fee-a995-6252d17a4474 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:19:31.568Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:19:31.061Z InvocationID=4bfb78fa-8573-4fee-a995-6252d17a4474 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:19:31.569Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:19:31.061Z InvocationID=4bfb78fa-8573-4fee-a995-6252d17a4474 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:19:31.569Z [qtp2045766957-16] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:19:31.061Z InvocationID=4bfb78fa-8573-4fee-a995-6252d17a4474 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:19:31.681Z [qtp2045766957-16] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=Create Resource - validateCapabilityTypesCreate ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2021-08-11T16:19:31.061Z InvocationID=4bfb78fa-8573-4fee-a995-6252d17a4474 ErrorCode=400 Capability Type tosca.capabilities.nfv.ContainerGroupable required is missing 2021-08-11T16:19:31.681Z [qtp2045766957-16] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=Create Resource - validateCapabilityTypesCreate ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2021-08-11T16:19:31.061Z InvocationID=4bfb78fa-8573-4fee-a995-6252d17a4474 ErrorCode=500 Operation towards database failed. 2021-08-11T16:19:31.685Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "osContainer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "" INVARIANT_UUID = "" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "400" DESC = "SVC4115: Error: Invalid Content. Missing Capability Type 'tosca.capabilities.nfv.ContainerGroupable'." TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.osContainer" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=Create Resource - validateCapabilityTypesCreate ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2021-08-11T16:19:31.061Z InvocationID=4bfb78fa-8573-4fee-a995-6252d17a4474 ErrorCode=500 lock resource with name Oscontainer 2021-08-11T16:19:31.974Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02a8b4d6-fc8c-499c-b4e8-6069019487d9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:19:31.974Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02a8b4d6-fc8c-499c-b4e8-6069019487d9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7793c9c5 2021-08-11T16:19:31.974Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02a8b4d6-fc8c-499c-b4e8-6069019487d9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:19:31.974Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02a8b4d6-fc8c-499c-b4e8-6069019487d9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:19:33.272Z [qtp2045766957-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "osContainer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "22f4c7f8-0bb9-4ad8-9b87-d29bee400d77" INVARIANT_UUID = "0966ebca-335d-4105-a3ab-14a10588da83" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.osContainer" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=Create Resource - validateCapabilityTypesCreate ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=22f4c7f8-0bb9-4ad8-9b87-d29bee400d77 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:31.061Z InvocationID=4bfb78fa-8573-4fee-a995-6252d17a4474 ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:19:33.273Z [qtp2045766957-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "osContainer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "22f4c7f8-0bb9-4ad8-9b87-d29bee400d77" INVARIANT_UUID = "0966ebca-335d-4105-a3ab-14a10588da83" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.osContainer" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=22f4c7f8-0bb9-4ad8-9b87-d29bee400d77 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:31.061Z InvocationID=4bfb78fa-8573-4fee-a995-6252d17a4474 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:19:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:19:33.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:19:33.574Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "osContainer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "22f4c7f8-0bb9-4ad8-9b87-d29bee400d77" INVARIANT_UUID = "0966ebca-335d-4105-a3ab-14a10588da83" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.osContainer" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=22f4c7f8-0bb9-4ad8-9b87-d29bee400d77 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:31.061Z InvocationID=4bfb78fa-8573-4fee-a995-6252d17a4474 ErrorCode=500 lock resource with id 6e831e69-0716-46c8-a1f3-3639226da634 2021-08-11T16:19:34.691Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d1a8c3cb-02ad-43a0-8a64-d542ce5b995d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:19:34.691Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d1a8c3cb-02ad-43a0-8a64-d542ce5b995d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@24a2010a 2021-08-11T16:19:34.691Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d1a8c3cb-02ad-43a0-8a64-d542ce5b995d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:19:34.691Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d1a8c3cb-02ad-43a0-8a64-d542ce5b995d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:19:35.220Z [qtp2045766957-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "osContainer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "22f4c7f8-0bb9-4ad8-9b87-d29bee400d77" INVARIANT_UUID = "0966ebca-335d-4105-a3ab-14a10588da83" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "6e831e69-0716-46c8-a1f3-3639226da634.assettoscatemplate" CURR_ARTIFACT_UUID = "6e831e69-0716-46c8-a1f3-3639226da634.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Oscontainer-template.yml,0,6e831e69-0716-46c8-a1f3-3639226da634.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ed588b3d-ebaa-4ec8-a7e8-595d75867ee5 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:31.061Z InvocationID=4bfb78fa-8573-4fee-a995-6252d17a4474 ErrorCode=500 Starting to create Csar for component osContainer 2021-08-11T16:19:35.353Z [qtp2045766957-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "osContainer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "22f4c7f8-0bb9-4ad8-9b87-d29bee400d77" INVARIANT_UUID = "0966ebca-335d-4105-a3ab-14a10588da83" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "6e831e69-0716-46c8-a1f3-3639226da634.assettoscatemplate" CURR_ARTIFACT_UUID = "6e831e69-0716-46c8-a1f3-3639226da634.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Oscontainer-template.yml,0,6e831e69-0716-46c8-a1f3-3639226da634.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ed588b3d-ebaa-4ec8-a7e8-595d75867ee5 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:31.061Z InvocationID=4bfb78fa-8573-4fee-a995-6252d17a4474 ErrorCode=500 Ended create Csar for component osContainer 2021-08-11T16:19:35.353Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "osContainer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "22f4c7f8-0bb9-4ad8-9b87-d29bee400d77" INVARIANT_UUID = "0966ebca-335d-4105-a3ab-14a10588da83" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "6e831e69-0716-46c8-a1f3-3639226da634.assettoscatemplate" CURR_ARTIFACT_UUID = "6e831e69-0716-46c8-a1f3-3639226da634.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Oscontainer-template.yml,0,6e831e69-0716-46c8-a1f3-3639226da634.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=ed588b3d-ebaa-4ec8-a7e8-595d75867ee5 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:31.061Z InvocationID=4bfb78fa-8573-4fee-a995-6252d17a4474 ErrorCode=500 lock resource with id 6e831e69-0716-46c8-a1f3-3639226da634 2021-08-11T16:19:36.969Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "osContainer" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "22f4c7f8-0bb9-4ad8-9b87-d29bee400d77" INVARIANT_UUID = "0966ebca-335d-4105-a3ab-14a10588da83" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "6e831e69-0716-46c8-a1f3-3639226da634.assettoscacsar" CURR_ARTIFACT_UUID = "6e831e69-0716-46c8-a1f3-3639226da634.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Oscontainer-csar.csar,0,6e831e69-0716-46c8-a1f3-3639226da634.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5908 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=8e24f841-57aa-4fbe-93f9-df6cc9e50508 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:31.061Z InvocationID=4bfb78fa-8573-4fee-a995-6252d17a4474 ErrorCode=500 LogTimestamp=2021-08-11T16:19:36.969403Z ResponseDesc=Created Exiting. 2021-08-11T16:19:36.970Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:19:36.987Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=003cf938-a000-46d6-82e1-60f97a083a46 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:19:36.987499Z InvocationID=bdbdf204-3672-414b-8db4-9f541faab704 LogTimestamp=2021-08-11T16:19:36.987552Z Entering 2021-08-11T16:19:36.988Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=003cf938-a000-46d6-82e1-60f97a083a46 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServerIPAddress=10.242.24.223 ServiceName=v1/catalog/upload/multipart ServiceInstanceID=null EntryTimestamp=2021-08-11T16:19:36.987499Z InvocationID=bdbdf204-3672-414b-8db4-9f541faab704 LogTimestamp=2021-08-11T16:19:36.987552Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:19:37.308Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:19:36.988Z InvocationID=d582dad7-4ea9-463b-9bec-1780f0736f9e PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:19:37.308Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:19:36.988Z InvocationID=d582dad7-4ea9-463b-9bec-1780f0736f9e PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:19:37.308Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:19:36.988Z InvocationID=d582dad7-4ea9-463b-9bec-1780f0736f9e PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:19:37.308Z [qtp2045766957-18] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:19:36.988Z InvocationID=d582dad7-4ea9-463b-9bec-1780f0736f9e PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:19:37.333Z [qtp2045766957-18] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=Create Resource - validateCapabilityTypesCreate ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2021-08-11T16:19:36.988Z InvocationID=d582dad7-4ea9-463b-9bec-1780f0736f9e ErrorCode=400 Capability Type tosca.capabilities.nfv.ContainerGroupable required is missing 2021-08-11T16:19:37.333Z [qtp2045766957-18] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=Create Resource - validateCapabilityTypesCreate ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2021-08-11T16:19:36.988Z InvocationID=d582dad7-4ea9-463b-9bec-1780f0736f9e ErrorCode=500 Operation towards database failed. 2021-08-11T16:19:37.335Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "osContainerGroup" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "" INVARIANT_UUID = "" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "400" DESC = "SVC4115: Error: Invalid Content. Missing Capability Type 'tosca.capabilities.nfv.ContainerGroupable'." TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.osContainerGroup" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=Create Resource - validateCapabilityTypesCreate ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2021-08-11T16:19:36.988Z InvocationID=d582dad7-4ea9-463b-9bec-1780f0736f9e ErrorCode=500 lock resource with name Oscontainergroup 2021-08-11T16:19:37.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a92c520-f007-4c21-81a0-75b16d2a1425 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:19:37.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a92c520-f007-4c21-81a0-75b16d2a1425 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@70e4f4fa 2021-08-11T16:19:37.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a92c520-f007-4c21-81a0-75b16d2a1425 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:19:37.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a92c520-f007-4c21-81a0-75b16d2a1425 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:19:38.949Z [qtp2045766957-18] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "osContainerGroup" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "15391b28-f6b3-41b8-bba0-c063042ae442" INVARIANT_UUID = "4d545a96-f5be-4f8e-a148-a1acba366295" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.osContainerGroup" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=Create Resource - validateCapabilityTypesCreate ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=15391b28-f6b3-41b8-bba0-c063042ae442 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:36.988Z InvocationID=d582dad7-4ea9-463b-9bec-1780f0736f9e ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:19:38.951Z [qtp2045766957-18] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "osContainerGroup" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "15391b28-f6b3-41b8-bba0-c063042ae442" INVARIANT_UUID = "4d545a96-f5be-4f8e-a148-a1acba366295" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.osContainerGroup" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=15391b28-f6b3-41b8-bba0-c063042ae442 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:36.988Z InvocationID=d582dad7-4ea9-463b-9bec-1780f0736f9e ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:19:39.572Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "osContainerGroup" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "15391b28-f6b3-41b8-bba0-c063042ae442" INVARIANT_UUID = "4d545a96-f5be-4f8e-a148-a1acba366295" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.Vdu.osContainerGroup" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=15391b28-f6b3-41b8-bba0-c063042ae442 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:36.988Z InvocationID=d582dad7-4ea9-463b-9bec-1780f0736f9e ErrorCode=500 lock resource with id addfde6e-e473-4bc2-b6b6-932325d01654 2021-08-11T16:19:40.951Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=51ad9498-7ff5-4c77-822f-774534740926 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:19:40.951Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=51ad9498-7ff5-4c77-822f-774534740926 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6fcc9d58 2021-08-11T16:19:40.951Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=51ad9498-7ff5-4c77-822f-774534740926 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:19:40.951Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=51ad9498-7ff5-4c77-822f-774534740926 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:19:41.383Z [qtp2045766957-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "osContainerGroup" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "15391b28-f6b3-41b8-bba0-c063042ae442" INVARIANT_UUID = "4d545a96-f5be-4f8e-a148-a1acba366295" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "addfde6e-e473-4bc2-b6b6-932325d01654.assettoscatemplate" CURR_ARTIFACT_UUID = "addfde6e-e473-4bc2-b6b6-932325d01654.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Oscontainergroup-template.yml,0,addfde6e-e473-4bc2-b6b6-932325d01654.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=538feb80-cf88-47fd-a420-76b529c923c5 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:36.988Z InvocationID=d582dad7-4ea9-463b-9bec-1780f0736f9e ErrorCode=500 Starting to create Csar for component osContainerGroup 2021-08-11T16:19:41.515Z [qtp2045766957-18] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "osContainerGroup" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "15391b28-f6b3-41b8-bba0-c063042ae442" INVARIANT_UUID = "4d545a96-f5be-4f8e-a148-a1acba366295" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "addfde6e-e473-4bc2-b6b6-932325d01654.assettoscatemplate" CURR_ARTIFACT_UUID = "addfde6e-e473-4bc2-b6b6-932325d01654.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Oscontainergroup-template.yml,0,addfde6e-e473-4bc2-b6b6-932325d01654.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=538feb80-cf88-47fd-a420-76b529c923c5 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:36.988Z InvocationID=d582dad7-4ea9-463b-9bec-1780f0736f9e ErrorCode=500 Ended create Csar for component osContainerGroup 2021-08-11T16:19:41.515Z [qtp2045766957-18] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "osContainerGroup" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "15391b28-f6b3-41b8-bba0-c063042ae442" INVARIANT_UUID = "4d545a96-f5be-4f8e-a148-a1acba366295" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "addfde6e-e473-4bc2-b6b6-932325d01654.assettoscatemplate" CURR_ARTIFACT_UUID = "addfde6e-e473-4bc2-b6b6-932325d01654.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-Oscontainergroup-template.yml,0,addfde6e-e473-4bc2-b6b6-932325d01654.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=538feb80-cf88-47fd-a420-76b529c923c5 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:36.988Z InvocationID=d582dad7-4ea9-463b-9bec-1780f0736f9e ErrorCode=500 lock resource with id addfde6e-e473-4bc2-b6b6-932325d01654 2021-08-11T16:19:43.266Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "osContainerGroup" RESOURCE_TYPE = "VFC" SERVICE_INSTANCE_ID = "15391b28-f6b3-41b8-bba0-c063042ae442" INVARIANT_UUID = "4d545a96-f5be-4f8e-a148-a1acba366295" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "addfde6e-e473-4bc2-b6b6-932325d01654.assettoscacsar" CURR_ARTIFACT_UUID = "addfde6e-e473-4bc2-b6b6-932325d01654.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-Oscontainergroup-csar.csar,0,addfde6e-e473-4bc2-b6b6-932325d01654.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=6278 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=86274f50-b13e-4c49-b7a1-1cb58f4e02ad TargetServiceName=null EntryTimestamp=2021-08-11T16:19:36.988Z InvocationID=d582dad7-4ea9-463b-9bec-1780f0736f9e ErrorCode=500 LogTimestamp=2021-08-11T16:19:43.266260Z ResponseDesc=Created Exiting. 2021-08-11T16:19:43.267Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:19:43.285Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=68dfba7e-5422-4fdc-99d6-8c040d43716a InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:19:43.285708Z InvocationID=23ddd71d-d9e1-4993-ac99-a5099cfb4b68 LogTimestamp=2021-08-11T16:19:43.285754Z Entering 2021-08-11T16:19:43.286Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=68dfba7e-5422-4fdc-99d6-8c040d43716a InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:19:43.285708Z InvocationID=23ddd71d-d9e1-4993-ac99-a5099cfb4b68 LogTimestamp=2021-08-11T16:19:43.285754Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:19:43.564Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:19:43.286Z InvocationID=a0ea0b6b-792a-412f-95c1-6e3a4df8b623 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with id 9ce3ffbd-c54f-433c-bd40-1621f1cfb26e 2021-08-11T16:19:43.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7bd57b2d-8184-44d5-81b0-b62b78ed69cf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:19:43.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7bd57b2d-8184-44d5-81b0-b62b78ed69cf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@678bc85 2021-08-11T16:19:43.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7bd57b2d-8184-44d5-81b0-b62b78ed69cf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:19:43.768Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7bd57b2d-8184-44d5-81b0-b62b78ed69cf PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:19:44.190Z [qtp2045766957-16] INFO o.o.s.b.c.l.LifecycleBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:19:43.286Z InvocationID=a0ea0b6b-792a-412f-95c1-6e3a4df8b623 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p owner of resource 9ce3ffbd-c54f-433c-bd40-1621f1cfb26e is jh0003 2021-08-11T16:19:44.231Z [qtp2045766957-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "c4978bcd-2fb3-4a80-94a5-bce31b25d8ae" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "2.0" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:19:43.286Z InvocationID=a0ea0b6b-792a-412f-95c1-6e3a4df8b623 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:19:44.231Z [qtp2045766957-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "c4978bcd-2fb3-4a80-94a5-bce31b25d8ae" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "2.0" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true TargetServiceName=null EntryTimestamp=2021-08-11T16:19:43.286Z InvocationID=a0ea0b6b-792a-412f-95c1-6e3a4df8b623 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:19:44.552Z [qtp2045766957-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "c4978bcd-2fb3-4a80-94a5-bce31b25d8ae" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "2.0" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=db93cf36-ceca-4541-b3ed-ee9d65f30b64 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:43.286Z InvocationID=a0ea0b6b-792a-412f-95c1-6e3a4df8b623 ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:19:44.552Z [qtp2045766957-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "Checkout" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "c4978bcd-2fb3-4a80-94a5-bce31b25d8ae" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "2.0" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "CERTIFIED" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" COMMENT = "update by import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=db93cf36-ceca-4541-b3ed-ee9d65f30b64 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:43.286Z InvocationID=a0ea0b6b-792a-412f-95c1-6e3a4df8b623 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:19:45.400Z [qtp2045766957-16] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "c4978bcd-2fb3-4a80-94a5-bce31b25d8ae" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "2.1" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VduCp" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=db93cf36-ceca-4541-b3ed-ee9d65f30b64 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:43.286Z InvocationID=a0ea0b6b-792a-412f-95c1-6e3a4df8b623 ErrorCode=500 lock resource with id d1494ff8-efab-4b32-a3d8-33343cb8cef1 2021-08-11T16:19:45.932Z [qtp2045766957-16] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "c4978bcd-2fb3-4a80-94a5-bce31b25d8ae" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "2.1" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKOUT" STATUS = "200" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VduCp" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=db93cf36-ceca-4541-b3ed-ee9d65f30b64 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:43.286Z InvocationID=a0ea0b6b-792a-412f-95c1-6e3a4df8b623 ErrorCode=500 owner of resource d1494ff8-efab-4b32-a3d8-33343cb8cef1 is jh0003 2021-08-11T16:19:46.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a714b22-cac2-493a-babd-3801e19c4488 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:19:46.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a714b22-cac2-493a-babd-3801e19c4488 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@32665711 2021-08-11T16:19:46.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a714b22-cac2-493a-babd-3801e19c4488 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:19:46.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a714b22-cac2-493a-babd-3801e19c4488 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:19:46.771Z [qtp2045766957-16] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "db93cf36-ceca-4541-b3ed-ee9d65f30b64" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "2.1" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=db93cf36-ceca-4541-b3ed-ee9d65f30b64 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:43.286Z InvocationID=a0ea0b6b-792a-412f-95c1-6e3a4df8b623 ErrorCode=500 owner of resource d1494ff8-efab-4b32-a3d8-33343cb8cef1 is jh0003 2021-08-11T16:19:46.771Z [qtp2045766957-16] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "db93cf36-ceca-4541-b3ed-ee9d65f30b64" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "2.1" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=db93cf36-ceca-4541-b3ed-ee9d65f30b64 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:43.286Z InvocationID=a0ea0b6b-792a-412f-95c1-6e3a4df8b623 ErrorCode=500 validate before certification change. resource name=VDU Cp, oldState=NOT_CERTIFIED_CHECKIN, owner userId=jh0003 2021-08-11T16:19:46.771Z [qtp2045766957-16] INFO o.o.s.b.c.l.CertificationChangeTransition AuditMessage=ACTION = "Checkin" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "db93cf36-ceca-4541-b3ed-ee9d65f30b64" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "2.1" CURR_VERSION = "2.1" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKOUT" CURR_STATE = "NOT_CERTIFIED_CHECKIN" COMMENT = "certification on import" STATUS = "200" DESC = "OK" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=db93cf36-ceca-4541-b3ed-ee9d65f30b64 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:43.286Z InvocationID=a0ea0b6b-792a-412f-95c1-6e3a4df8b623 ErrorCode=500 start performing certification change for resource d1494ff8-efab-4b32-a3d8-33343cb8cef1 2021-08-11T16:19:47.230Z [qtp2045766957-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "db93cf36-ceca-4541-b3ed-ee9d65f30b64" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "" CURR_VERSION = "2.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "ffde7336-c038-416b-82fa-87114fbc8a28.assettoscatemplate" CURR_ARTIFACT_UUID = "ffde7336-c038-416b-82fa-87114fbc8a28.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduCp-template.yml,0,ffde7336-c038-416b-82fa-87114fbc8a28.assettoscatemplate,2,b309f831-be7f-4ad1-b936-1a48253dd3cc" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b0ac3978-8b61-44c1-90d8-ce33b47b378c TargetServiceName=null EntryTimestamp=2021-08-11T16:19:43.286Z InvocationID=a0ea0b6b-792a-412f-95c1-6e3a4df8b623 ErrorCode=500 Starting to create Csar for component VDU Cp 2021-08-11T16:19:47.381Z [qtp2045766957-16] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "db93cf36-ceca-4541-b3ed-ee9d65f30b64" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "" CURR_VERSION = "2.1" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "NOT_CERTIFIED_CHECKIN" PREV_ARTIFACT_UUID = "ffde7336-c038-416b-82fa-87114fbc8a28.assettoscatemplate" CURR_ARTIFACT_UUID = "ffde7336-c038-416b-82fa-87114fbc8a28.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VduCp-template.yml,0,ffde7336-c038-416b-82fa-87114fbc8a28.assettoscatemplate,2,b309f831-be7f-4ad1-b936-1a48253dd3cc" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=b0ac3978-8b61-44c1-90d8-ce33b47b378c TargetServiceName=null EntryTimestamp=2021-08-11T16:19:43.286Z InvocationID=a0ea0b6b-792a-412f-95c1-6e3a4df8b623 ErrorCode=500 Ended create Csar for component VDU Cp 2021-08-11T16:19:48.177Z [qtp2045766957-16] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "db93cf36-ceca-4541-b3ed-ee9d65f30b64" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "2.1" CURR_VERSION = "3.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=8101b7d3-f8b1-4fd7-94a2-1dff8c1f71b2 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:43.286Z InvocationID=a0ea0b6b-792a-412f-95c1-6e3a4df8b623 ErrorCode=500 The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:19:48.178Z [qtp2045766957-16] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "db93cf36-ceca-4541-b3ed-ee9d65f30b64" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "2.1" CURR_VERSION = "3.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=8101b7d3-f8b1-4fd7-94a2-1dff8c1f71b2 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:43.286Z InvocationID=a0ea0b6b-792a-412f-95c1-6e3a4df8b623 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:19:48.179Z [qtp2045766957-16] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "db93cf36-ceca-4541-b3ed-ee9d65f30b64" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "2.1" CURR_VERSION = "3.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=8101b7d3-f8b1-4fd7-94a2-1dff8c1f71b2 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:43.286Z InvocationID=a0ea0b6b-792a-412f-95c1-6e3a4df8b623 ErrorCode=500 unlock component d1494ff8-efab-4b32-a3d8-33343cb8cef1 2021-08-11T16:19:48.179Z [qtp2045766957-16] INFO o.o.s.b.c.l.LifecycleBusinessLogic AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "db93cf36-ceca-4541-b3ed-ee9d65f30b64" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "2.1" CURR_VERSION = "3.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=8101b7d3-f8b1-4fd7-94a2-1dff8c1f71b2 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:43.286Z InvocationID=a0ea0b6b-792a-412f-95c1-6e3a4df8b623 ErrorCode=500 During change state, another component d1494ff8-efab-4b32-a3d8-33343cb8cef1 has been created/updated 2021-08-11T16:19:48.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:19:48.317Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:19:48.776Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "CertificationSuccess" RESOURCE_NAME = "VDU Cp" RESOURCE_TYPE = "Resource" SERVICE_INSTANCE_ID = "db93cf36-ceca-4541-b3ed-ee9d65f30b64" INVARIANT_UUID = "585eb3f7-4d87-4db8-a345-63b120b1449b" PREV_VERSION = "2.1" CURR_VERSION = "3.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "NOT_CERTIFIED_CHECKIN" CURR_STATE = "CERTIFIED" COMMENT = "certification on import" STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=5490 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=8101b7d3-f8b1-4fd7-94a2-1dff8c1f71b2 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:43.286Z InvocationID=a0ea0b6b-792a-412f-95c1-6e3a4df8b623 ErrorCode=500 LogTimestamp=2021-08-11T16:19:48.776441Z ResponseDesc=Created Exiting. 2021-08-11T16:19:48.777Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:19:48.790Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=3ccb79a3-e984-4de8-ac8f-1d3c0421f155 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:19:48.789876Z InvocationID=0fa416bf-7098-48d3-ad53-b70674914cb0 LogTimestamp=2021-08-11T16:19:48.789908Z Entering 2021-08-11T16:19:48.790Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=3ccb79a3-e984-4de8-ac8f-1d3c0421f155 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/upload/multipart ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:19:48.789876Z InvocationID=0fa416bf-7098-48d3-ad53-b70674914cb0 LogTimestamp=2021-08-11T16:19:48.789908Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 2021-08-11T16:19:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:19:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:19:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:19:49.947Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:19:48.790Z InvocationID=6e0df88b-3d90-421f-b077-eaea6f071968 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p LifecycleState cannot be defined by user. This field will be overridden by the application 2021-08-11T16:19:49.947Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:19:48.790Z InvocationID=6e0df88b-3d90-421f-b077-eaea6f071968 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:19:49.947Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:19:48.790Z InvocationID=6e0df88b-3d90-421f-b077-eaea6f071968 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Creator cannot be defined by user. This field will be overridden by the application 2021-08-11T16:19:49.947Z [qtp2045766957-14] INFO o.o.s.b.c.i.ComponentBusinessLogic RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:19:48.790Z InvocationID=6e0df88b-3d90-421f-b077-eaea6f071968 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p Is Highest Version cannot be defined by user. This field will be overridden by the application 2021-08-11T16:19:50.055Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true EntryTimestamp=2021-08-11T16:19:48.790Z InvocationID=6e0df88b-3d90-421f-b077-eaea6f071968 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p lock resource with name VirtualCp 2021-08-11T16:19:50.267Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b682eadb-5cd8-47ea-a8e1-9a6879607f7a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:19:50.267Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b682eadb-5cd8-47ea-a8e1-9a6879607f7a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@65723683 2021-08-11T16:19:50.267Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b682eadb-5cd8-47ea-a8e1-9a6879607f7a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:19:50.267Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b682eadb-5cd8-47ea-a8e1-9a6879607f7a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:19:52.462Z [qtp2045766957-14] INFO o.o.s.b.c.d.e.DmaapClientFactory AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Virtual Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "48feb97d-c2df-492d-b61c-8ff2c4e9c150" INVARIANT_UUID = "3f00cd5b-7564-4dc3-adee-5f02caa31f43" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VirtualCp" RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=48feb97d-c2df-492d-b61c-8ff2c4e9c150 EntryTimestamp=2021-08-11T16:19:48.790Z InvocationID=6e0df88b-3d90-421f-b077-eaea6f071968 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p The DmaapProducerConfiguration is DmaapProducerConfiguration(active=true, hosts=olsd004.com:3905, consumerGroup=sdc-AUTO-1628697733, consumerId=sdc-AUTO1-1628697733, timeoutMs=15000, limit=1, pollingInterval=2, topic=SDC-FACADE-NOTIF-v1, latitude=32.109333, longitude=34.855499, version=1.0, serviceName=dmaap.com:3905/events, environment=TEST, partner=BOT_R, routeOffer=MR1, protocol=null, contenttype=application/json, dme2TraceOn=true, aftEnvironment=null, aftDme2ConnectionTimeoutMs=15000, aftDme2RoundtripTimeoutMs=240000, aftDme2ReadTimeoutMs=50000, dme2preferredRouterFilePath=null, credential=Credential [username=user1@sdc.com, password=password==], timeLimitForNotificationHandleMs=120000, aftDme2SslEnable=true, aftDme2ClientIgnoreSslConfig=false, aftDme2ClientKeystore=/app/jetty/etc/truststore, aftDme2ClientKeystorePassword=dmaap_truststore_pwd, aftDme2ClientSslCertAlias=certman) 2021-08-11T16:19:52.462Z [qtp2045766957-14] ERROR o.o.s.be.catalog.impl.DmaapProducer AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Virtual Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "48feb97d-c2df-492d-b61c-8ff2c4e9c150" INVARIANT_UUID = "3f00cd5b-7564-4dc3-adee-5f02caa31f43" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VirtualCp" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=48feb97d-c2df-492d-b61c-8ff2c4e9c150 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:48.790Z InvocationID=6e0df88b-3d90-421f-b077-eaea6f071968 ErrorCode=500 Failed to connect to topic . Exeption Input byte array has wrong 4-byte ending unit 2021-08-11T16:19:53.274Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a1bd5c5-70bd-460b-ae73-f549d418e730 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:19:53.274Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a1bd5c5-70bd-460b-ae73-f549d418e730 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6edf4128 2021-08-11T16:19:53.274Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a1bd5c5-70bd-460b-ae73-f549d418e730 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:19:53.275Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a1bd5c5-70bd-460b-ae73-f549d418e730 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:19:54.168Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ResourceImport" RESOURCE_NAME = "Virtual Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "48feb97d-c2df-492d-b61c-8ff2c4e9c150" INVARIANT_UUID = "3f00cd5b-7564-4dc3-adee-5f02caa31f43" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "Jimmy Hendrix(jh0003)" PREV_STATE = "" CURR_STATE = "CERTIFIED" STATUS = "201" DESC = "OK" TOSCA_NODE_TYPE = "tosca.nodes.nfv.VirtualCp" RequestID=null PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=48feb97d-c2df-492d-b61c-8ff2c4e9c150 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:48.790Z InvocationID=6e0df88b-3d90-421f-b077-eaea6f071968 ErrorCode=500 lock resource with id 03498634-8810-42ca-b380-ae2f197944a7 2021-08-11T16:19:56.080Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=741ea9c8-4341-4670-89be-c3c0d5e6e741 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:19:56.080Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=741ea9c8-4341-4670-89be-c3c0d5e6e741 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2d0054e2 2021-08-11T16:19:56.080Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=741ea9c8-4341-4670-89be-c3c0d5e6e741 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:19:56.080Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=741ea9c8-4341-4670-89be-c3c0d5e6e741 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:19:56.383Z [qtp2045766957-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Virtual Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "48feb97d-c2df-492d-b61c-8ff2c4e9c150" INVARIANT_UUID = "3f00cd5b-7564-4dc3-adee-5f02caa31f43" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "03498634-8810-42ca-b380-ae2f197944a7.assettoscatemplate" CURR_ARTIFACT_UUID = "03498634-8810-42ca-b380-ae2f197944a7.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VirtualCp-template.yml,0,03498634-8810-42ca-b380-ae2f197944a7.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=STARTED ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=5f875611-0692-4dfd-ae9e-9e057e203201 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:48.790Z InvocationID=6e0df88b-3d90-421f-b077-eaea6f071968 ErrorCode=500 Starting to create Csar for component Virtual Cp 2021-08-11T16:19:56.638Z [qtp2045766957-14] INFO org.openecomp.sdc.be.tosca.CsarUtils AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Virtual Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "48feb97d-c2df-492d-b61c-8ff2c4e9c150" INVARIANT_UUID = "3f00cd5b-7564-4dc3-adee-5f02caa31f43" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "03498634-8810-42ca-b380-ae2f197944a7.assettoscatemplate" CURR_ARTIFACT_UUID = "03498634-8810-42ca-b380-ae2f197944a7.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VirtualCp-template.yml,0,03498634-8810-42ca-b380-ae2f197944a7.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=5f875611-0692-4dfd-ae9e-9e057e203201 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:48.790Z InvocationID=6e0df88b-3d90-421f-b077-eaea6f071968 ErrorCode=500 Ended create Csar for component Virtual Cp 2021-08-11T16:19:56.639Z [qtp2045766957-14] INFO o.o.s.b.m.o.impl.GraphLockOperation AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Virtual Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "48feb97d-c2df-492d-b61c-8ff2c4e9c150" INVARIANT_UUID = "3f00cd5b-7564-4dc3-adee-5f02caa31f43" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "03498634-8810-42ca-b380-ae2f197944a7.assettoscatemplate" CURR_ARTIFACT_UUID = "03498634-8810-42ca-b380-ae2f197944a7.assettoscatemplate" ARTIFACT_DATA = "TOSCA,'assettoscatemplate',TOSCA_TEMPLATE,resource-VirtualCp-template.yml,0,03498634-8810-42ca-b380-ae2f197944a7.assettoscatemplate,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR RequestID=null PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=5f875611-0692-4dfd-ae9e-9e057e203201 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:48.790Z InvocationID=6e0df88b-3d90-421f-b077-eaea6f071968 ErrorCode=500 lock resource with id 03498634-8810-42ca-b380-ae2f197944a7 2021-08-11T16:19:58.385Z [qtp2045766957-14] INFO o.o.l.f.base.AbstractAuditLogFilter AuditMessage=ACTION = "ArtifactPayloadUpdate" RESOURCE_NAME = "Virtual Cp" RESOURCE_TYPE = "CP" SERVICE_INSTANCE_ID = "48feb97d-c2df-492d-b61c-8ff2c4e9c150" INVARIANT_UUID = "3f00cd5b-7564-4dc3-adee-5f02caa31f43" PREV_VERSION = "" CURR_VERSION = "1.0" MODIFIER = "(UNKNOWN)" PREV_STATE = "" CURR_STATE = "CERTIFIED" PREV_ARTIFACT_UUID = "03498634-8810-42ca-b380-ae2f197944a7.assettoscacsar" CURR_ARTIFACT_UUID = "03498634-8810-42ca-b380-ae2f197944a7.assettoscacsar" ARTIFACT_DATA = "TOSCA,'assettoscacsar',TOSCA_CSAR,resource-VirtualCp-csar.csar,0,03498634-8810-42ca-b380-ae2f197944a7.assettoscacsar,0 " STATUS = "200" DESC = "OK" SupportablityAction=GENERATE CSAR ResponseCode=201 RequestID=null ElapsedTime=9595 PartnerName=jh0003 auditOn=true SupportablityStatus=COMPLETE ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE TargetEntity=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/upload/multipart?createNewVersion=true ServiceInstanceID=9818399b-09fa-4e60-be9e-d60017941881 TargetServiceName=null EntryTimestamp=2021-08-11T16:19:48.790Z InvocationID=6e0df88b-3d90-421f-b077-eaea6f071968 ErrorCode=500 LogTimestamp=2021-08-11T16:19:58.385233Z ResponseDesc=Created Exiting. 2021-08-11T16:19:58.386Z [qtp2045766957-14] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/upload/multipart HTTP/1.1 SC="201" 2021-08-11T16:19:58.405Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=86b505c9-df70-467e-b2d5-45fc914a92f1 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/uploadType/grouptypes ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:19:58.405055Z InvocationID=b439efb6-b763-4ffc-99fc-b0f02de0129d LogTimestamp=2021-08-11T16:19:58.405094Z Entering 2021-08-11T16:19:58.405Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=86b505c9-df70-467e-b2d5-45fc914a92f1 InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/uploadType/grouptypes ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:19:58.405055Z InvocationID=b439efb6-b763-4ffc-99fc-b0f02de0129d LogTimestamp=2021-08-11T16:19:58.405094Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/uploadType/grouptypes HTTP/1.1 2021-08-11T16:19:58.432Z [qtp2045766957-18] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/grouptypes EntryTimestamp=2021-08-11T16:19:58.405Z InvocationID=f096049f-beb5-4bfa-8dc7-9cd8fbc01fc8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send GROUP_TYPE : tosca.groups.Root to dao for create 2021-08-11T16:19:58.469Z [qtp2045766957-18] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/grouptypes EntryTimestamp=2021-08-11T16:19:58.405Z InvocationID=f096049f-beb5-4bfa-8dc7-9cd8fbc01fc8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send GROUP_TYPE : org.openecomp.groups.heat.HeatStack to dao for create 2021-08-11T16:19:58.547Z [qtp2045766957-18] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/grouptypes EntryTimestamp=2021-08-11T16:19:58.405Z InvocationID=f096049f-beb5-4bfa-8dc7-9cd8fbc01fc8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send GROUP_TYPE : org.openecomp.groups.VfModule to dao for create 2021-08-11T16:19:58.604Z [qtp2045766957-18] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/grouptypes EntryTimestamp=2021-08-11T16:19:58.405Z InvocationID=f096049f-beb5-4bfa-8dc7-9cd8fbc01fc8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send GROUP_TYPE : org.openecomp.groups.NetworkCollection to dao for create 2021-08-11T16:19:58.660Z [qtp2045766957-18] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/grouptypes EntryTimestamp=2021-08-11T16:19:58.405Z InvocationID=f096049f-beb5-4bfa-8dc7-9cd8fbc01fc8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send GROUP_TYPE : org.openecomp.groups.VfcInstanceGroup to dao for create 2021-08-11T16:19:58.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c2684e17-4710-48b7-bed7-faa692319b51 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:19:58.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c2684e17-4710-48b7-bed7-faa692319b51 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2dde408c 2021-08-11T16:19:58.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c2684e17-4710-48b7-bed7-faa692319b51 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:19:58.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c2684e17-4710-48b7-bed7-faa692319b51 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:19:59.027Z [qtp2045766957-18] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/grouptypes EntryTimestamp=2021-08-11T16:19:58.405Z InvocationID=f096049f-beb5-4bfa-8dc7-9cd8fbc01fc8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send GROUP_TYPE : tosca.groups.nfv.PlacementGroup to dao for create 2021-08-11T16:19:59.333Z [qtp2045766957-18] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/grouptypes EntryTimestamp=2021-08-11T16:19:58.405Z InvocationID=f096049f-beb5-4bfa-8dc7-9cd8fbc01fc8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send GROUP_TYPE : tosca.groups.nfv.NsPlacementGroup to dao for create 2021-08-11T16:19:59.521Z [qtp2045766957-18] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/grouptypes EntryTimestamp=2021-08-11T16:19:58.405Z InvocationID=f096049f-beb5-4bfa-8dc7-9cd8fbc01fc8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send GROUP_TYPE : tosca.groups.nfv.VNFFG to dao for create 2021-08-11T16:19:59.534Z [qtp2045766957-18] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/grouptypes EntryTimestamp=2021-08-11T16:19:58.405Z InvocationID=f096049f-beb5-4bfa-8dc7-9cd8fbc01fc8 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p all GroupTypeDataDefinition [uniqueId=tosca.groups.nfv.VNFFG.1.0.grouptype, type=tosca.groups.nfv.VNFFG, name=NFV VNF Forwarding Graph, icon=group-nfv-placementgroup, version=1.0, members=[tosca.nodes.nfv.NFP, tosca.nodes.nfv.VNF, tosca.nodes.nfv.NS, tosca.nodes.nfv.NsVirtualLink, tosca.nodes.nfv.NfpPositionElement], metadata=null, description=the VNFFG group type describes a topology of the NS or a portion of the NS, and optionally forwarding rules, applicable to the traffic conveyed over this topology, creationTime=1628698799525, modificationTime=1628698799525] [properties=[PropertyDefinition [ PropertyDataDefinition(definition=false, hidden=false, uniqueId=tosca.groups.nfv.VNFFG.1.0.grouptype.description, type=string, required=true, defaultValue=null, description=Human readable description of the group, schema=null, password=false, name=description, value=null, label=null, immutable=false, mappedToComponentProperty=true, toscaGetFunctionType=null, inputPath=null, status=null, inputId=null, instanceUniqueId=null, propertyId=null, parentPropertyType=null, subPropertyInputPath=null, annotations=null, parentUniqueId=null, getInputValues=null, isDeclaredListInput=false, getPolicyValues=null, propertyConstraints=null, metadata=null), name=description, constraints=null]]], capabilities={}] were created successfully!!! 2021-08-11T16:20:01.115Z [qtp2045766957-18] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=2709 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/grouptypes EntryTimestamp=2021-08-11T16:19:58.405Z InvocationID=f096049f-beb5-4bfa-8dc7-9cd8fbc01fc8 LogTimestamp=2021-08-11T16:20:01.114947Z ResponseDesc=Created Exiting. 2021-08-11T16:20:01.115Z [qtp2045766957-18] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/grouptypes HTTP/1.1 SC="201" 2021-08-11T16:20:01.168Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter RequestID=2481abaf-53ac-4f40-8e1e-c2b37246c14b InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS RemoteHost=10.242.110.138 ServiceName=v1/catalog/uploadType/policytypes ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:20:01.168118Z InvocationID=4b2500d2-9ad2-4ace-a979-12f7374dc1eb LogTimestamp=2021-08-11T16:20:01.168175Z Entering 2021-08-11T16:20:01.168Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=2481abaf-53ac-4f40-8e1e-c2b37246c14b InstanceID=d7f9de6b-0419-40e9-ab1b-5a00c5f0a4da ElapsedTime=0 Severity=0 PartnerName=testName ServerFQDN=dev-sdc-be-997dd74c7-dmm7p ClientIPAddress=10.242.110.138 StatusCode=INPROGRESS userId=jh0003 RemoteHost=10.242.110.138 ServiceName=v1/catalog/uploadType/policytypes ServerIPAddress=10.242.24.223 ServiceInstanceID=null EntryTimestamp=2021-08-11T16:20:01.168118Z InvocationID=4b2500d2-9ad2-4ace-a979-12f7374dc1eb LogTimestamp=2021-08-11T16:20:01.168175Z localAddr=10.242.24.223 remoteAddr=10.242.110.138 POST /sdc2/rest/v1/catalog/uploadType/policytypes HTTP/1.1 2021-08-11T16:20:01.200Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2021-08-11T16:20:01.168Z InvocationID=7df9a570-c385-4083-a4aa-a51db4c7e43f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send POLICY_TYPE : tosca.policies.Root to dao for create 2021-08-11T16:20:01.221Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2021-08-11T16:20:01.168Z InvocationID=7df9a570-c385-4083-a4aa-a51db4c7e43f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send POLICY_TYPE : tosca.policies.Placement to dao for create 2021-08-11T16:20:01.280Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2021-08-11T16:20:01.168Z InvocationID=7df9a570-c385-4083-a4aa-a51db4c7e43f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send POLICY_TYPE : tosca.policies.Scaling to dao for create 2021-08-11T16:20:01.287Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2021-08-11T16:20:01.168Z InvocationID=7df9a570-c385-4083-a4aa-a51db4c7e43f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send POLICY_TYPE : tosca.policies.Update to dao for create 2021-08-11T16:20:01.295Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2021-08-11T16:20:01.168Z InvocationID=7df9a570-c385-4083-a4aa-a51db4c7e43f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send POLICY_TYPE : tosca.policies.Performance to dao for create 2021-08-11T16:20:01.303Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2021-08-11T16:20:01.168Z InvocationID=7df9a570-c385-4083-a4aa-a51db4c7e43f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send POLICY_TYPE : org.openecomp.policies.placement.Antilocate to dao for create 2021-08-11T16:20:01.325Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2021-08-11T16:20:01.168Z InvocationID=7df9a570-c385-4083-a4aa-a51db4c7e43f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send POLICY_TYPE : org.openecomp.policies.placement.Colocate to dao for create 2021-08-11T16:20:01.378Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2021-08-11T16:20:01.168Z InvocationID=7df9a570-c385-4083-a4aa-a51db4c7e43f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send POLICY_TYPE : org.openecomp.policies.placement.valet.Diversity to dao for create 2021-08-11T16:20:01.397Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2021-08-11T16:20:01.168Z InvocationID=7df9a570-c385-4083-a4aa-a51db4c7e43f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send POLICY_TYPE : org.openecomp.policies.placement.valet.Exclusivity to dao for create 2021-08-11T16:20:01.410Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2021-08-11T16:20:01.168Z InvocationID=7df9a570-c385-4083-a4aa-a51db4c7e43f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send POLICY_TYPE : org.openecomp.policies.placement.valet.Affinity to dao for create 2021-08-11T16:20:01.472Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2021-08-11T16:20:01.168Z InvocationID=7df9a570-c385-4083-a4aa-a51db4c7e43f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send POLICY_TYPE : org.openecomp.policies.scaling.Fixed to dao for create 2021-08-11T16:20:01.487Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2021-08-11T16:20:01.168Z InvocationID=7df9a570-c385-4083-a4aa-a51db4c7e43f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send POLICY_TYPE : org.openecomp.policies.External to dao for create 2021-08-11T16:20:01.505Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2021-08-11T16:20:01.168Z InvocationID=7df9a570-c385-4083-a4aa-a51db4c7e43f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send POLICY_TYPE : tosca.policies.nfv.InstantiationLevels to dao for create 2021-08-11T16:20:01.878Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2021-08-11T16:20:01.168Z InvocationID=7df9a570-c385-4083-a4aa-a51db4c7e43f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send POLICY_TYPE : tosca.policies.nfv.VduInstantiationLevels to dao for create 2021-08-11T16:20:02.077Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=20c1713a-f439-43c3-a3c6-9bad9ae7117d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:20:02.077Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=20c1713a-f439-43c3-a3c6-9bad9ae7117d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@59ecbd9a 2021-08-11T16:20:02.077Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=20c1713a-f439-43c3-a3c6-9bad9ae7117d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:20:02.077Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=20c1713a-f439-43c3-a3c6-9bad9ae7117d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:20:02.261Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2021-08-11T16:20:01.168Z InvocationID=7df9a570-c385-4083-a4aa-a51db4c7e43f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send POLICY_TYPE : tosca.policies.nfv.VirtualLinkInstantiationLevels to dao for create 2021-08-11T16:20:02.279Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2021-08-11T16:20:01.168Z InvocationID=7df9a570-c385-4083-a4aa-a51db4c7e43f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send POLICY_TYPE : tosca.policies.nfv.ScalingAspects to dao for create 2021-08-11T16:20:02.298Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2021-08-11T16:20:01.168Z InvocationID=7df9a570-c385-4083-a4aa-a51db4c7e43f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send POLICY_TYPE : tosca.policies.nfv.VduScalingAspectDeltas to dao for create 2021-08-11T16:20:02.320Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2021-08-11T16:20:01.168Z InvocationID=7df9a570-c385-4083-a4aa-a51db4c7e43f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send POLICY_TYPE : tosca.policies.nfv.VirtualLinkBitrateScalingAspectDeltas to dao for create 2021-08-11T16:20:02.347Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2021-08-11T16:20:01.168Z InvocationID=7df9a570-c385-4083-a4aa-a51db4c7e43f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send POLICY_TYPE : tosca.policies.nfv.VduInitialDelta to dao for create 2021-08-11T16:20:02.365Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2021-08-11T16:20:01.168Z InvocationID=7df9a570-c385-4083-a4aa-a51db4c7e43f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send POLICY_TYPE : tosca.policies.nfv.VirtualLinkBitrateInitialDelta to dao for create 2021-08-11T16:20:02.553Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2021-08-11T16:20:01.168Z InvocationID=7df9a570-c385-4083-a4aa-a51db4c7e43f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send POLICY_TYPE : tosca.policies.nfv.SecurityGroupRule to dao for create 2021-08-11T16:20:02.606Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2021-08-11T16:20:01.168Z InvocationID=7df9a570-c385-4083-a4aa-a51db4c7e43f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send POLICY_TYPE : tosca.policies.nfv.SupportedVnfInterface to dao for create 2021-08-11T16:20:03.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:20:03.317Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:20:03.856Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2021-08-11T16:20:01.168Z InvocationID=7df9a570-c385-4083-a4aa-a51db4c7e43f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send POLICY_TYPE : tosca.policies.nfv.AffinityRule to dao for create 2021-08-11T16:20:04.253Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:20:04.610Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2021-08-11T16:20:01.168Z InvocationID=7df9a570-c385-4083-a4aa-a51db4c7e43f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send POLICY_TYPE : tosca.policies.nfv.AntiAffinityRule to dao for create 2021-08-11T16:20:04.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0d38a94-1411-4469-9f6b-7489eedddee4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:20:04.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0d38a94-1411-4469-9f6b-7489eedddee4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4d2e618e 2021-08-11T16:20:04.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0d38a94-1411-4469-9f6b-7489eedddee4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:20:04.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0d38a94-1411-4469-9f6b-7489eedddee4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:20:05.742Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2021-08-11T16:20:01.168Z InvocationID=7df9a570-c385-4083-a4aa-a51db4c7e43f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send POLICY_TYPE : tosca.policies.nfv.NsAffinityRule to dao for create 2021-08-11T16:20:06.892Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2021-08-11T16:20:01.168Z InvocationID=7df9a570-c385-4083-a4aa-a51db4c7e43f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send POLICY_TYPE : tosca.policies.nfv.NsAntiAffinityRule to dao for create 2021-08-11T16:20:07.005Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2021-08-11T16:20:01.168Z InvocationID=7df9a570-c385-4083-a4aa-a51db4c7e43f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send POLICY_TYPE : tosca.policies.nfv.Abstract.SecurityGroupRule to dao for create 2021-08-11T16:20:07.286Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2021-08-11T16:20:01.168Z InvocationID=7df9a570-c385-4083-a4aa-a51db4c7e43f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send POLICY_TYPE : tosca.policies.nfv.NsSecurityGroupRule to dao for create 2021-08-11T16:20:07.480Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2021-08-11T16:20:01.168Z InvocationID=7df9a570-c385-4083-a4aa-a51db4c7e43f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send POLICY_TYPE : tosca.policies.nfv.NfpRule to dao for create 2021-08-11T16:20:07.896Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2021-08-11T16:20:01.168Z InvocationID=7df9a570-c385-4083-a4aa-a51db4c7e43f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send POLICY_TYPE : tosca.policies.nfv.NsMonitoring to dao for create 2021-08-11T16:20:08.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a7e50de-b21b-45ef-86a8-c2e2e9317864 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:20:08.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a7e50de-b21b-45ef-86a8-c2e2e9317864 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@62268b8d 2021-08-11T16:20:08.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a7e50de-b21b-45ef-86a8-c2e2e9317864 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:20:08.169Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a7e50de-b21b-45ef-86a8-c2e2e9317864 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:20:08.292Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2021-08-11T16:20:01.168Z InvocationID=7df9a570-c385-4083-a4aa-a51db4c7e43f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p send POLICY_TYPE : tosca.policies.nfv.VnfMonitoring to dao for create 2021-08-11T16:20:08.305Z [qtp2045766957-16] INFO o.o.s.b.c.impl.CommonImportManager RequestID=null ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2021-08-11T16:20:01.168Z InvocationID=7df9a570-c385-4083-a4aa-a51db4c7e43f PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p all PolicyTypeDataDefinition [uniqueId=tosca.policies.nfv.VnfMonitoring.1.0.policytype, type=tosca.policies.nfv.VnfMonitoring, name=NFV VnfMonitoring, icon=policy-nfv-anti-affinityrule, version=1.0, targets=[tosca.nodes.nfv.VNF], metadata=null, description=Policy type is used to identify information to be monitored during the lifetime of a VNF instance as defined in ETSI GS NFV-IFA 014 [2]., creationTime=1628698808295, modificationTime=1628698808295] [ properties=[PropertyDefinition [ PropertyDataDefinition(definition=false, hidden=false, uniqueId=tosca.policies.nfv.VnfMonitoring.1.0.policytype.vnf_monitoring_parameters, type=list, required=true, defaultValue=null, description=Specifies a virtualised resource related performance metric to be monitored on the NS level., schema=SchemaDefinition [derivedFrom='null, constraints=null, properties=null, property=PropertyDefinition [ PropertyDataDefinition(definition=false, hidden=false, uniqueId=null, type=tosca.datatypes.nfv.VnfMonitoringParameter, required=false, defaultValue=null, description=null, schema=null, password=false, name=null, value=null, label=null, immutable=false, mappedToComponentProperty=true, toscaGetFunctionType=null, inputPath=null, status=null, inputId=null, instanceUniqueId=null, propertyId=null, parentPropertyType=null, subPropertyInputPath=null, annotations=null, parentUniqueId=null, getInputValues=null, isDeclaredListInput=false, getPolicyValues=null, propertyConstraints=null, metadata=null), name=null, constraints=null]]], password=false, name=vnf_monitoring_parameters, value=null, label=null, immutable=false, mappedToComponentProperty=true, toscaGetFunctionType=null, inputPath=null, status=null, inputId=null, instanceUniqueId=null, propertyId=null, parentPropertyType=null, subPropertyInputPath=null, annotations=null, parentUniqueId=null, getInputValues=null, isDeclaredListInput=false, getPolicyValues=null, propertyConstraints=[{"minLength":1}], metadata=null), name=vnf_monitoring_parameters, constraints=[org.openecomp.sdc.be.model.tosca.constraints.MinLengthConstraint@5b974c2e]]]] ] were created successfully!!! 2021-08-11T16:20:09.179Z [qtp2045766957-16] INFO o.o.l.f.base.AbstractAuditLogFilter ResponseCode=201 RequestID=null ElapsedTime=8010 PartnerName=jh0003 auditOn=true ServerFQDN=dev-sdc-be-997dd74c7-dmm7p StatusCode=COMPLETE ServerIPAddress=10.242.24.223 ServiceName=/v1/catalog/uploadType/policytypes EntryTimestamp=2021-08-11T16:20:01.168Z InvocationID=7df9a570-c385-4083-a4aa-a51db4c7e43f LogTimestamp=2021-08-11T16:20:09.178293Z ResponseDesc=Created Exiting. 2021-08-11T16:20:09.179Z [qtp2045766957-16] INFO o.o.sdc.be.filters.BeServletFilter RequestID=null POST /sdc2/rest/v1/catalog/uploadType/policytypes HTTP/1.1 SC="201" 2021-08-11T16:20:10.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4da397c7-d26c-4037-b194-797540719dcc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:20:10.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4da397c7-d26c-4037-b194-797540719dcc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3da36d65 2021-08-11T16:20:10.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4da397c7-d26c-4037-b194-797540719dcc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:20:10.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4da397c7-d26c-4037-b194-797540719dcc PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:20:13.689Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc2eed41-60f4-4d5c-8fce-77b421df27d4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:20:13.689Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc2eed41-60f4-4d5c-8fce-77b421df27d4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@19f1726 2021-08-11T16:20:13.689Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc2eed41-60f4-4d5c-8fce-77b421df27d4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:20:13.689Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc2eed41-60f4-4d5c-8fce-77b421df27d4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:20:16.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fc5aeb23-5a48-43c0-b138-2c47f36a59e0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:20:16.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fc5aeb23-5a48-43c0-b138-2c47f36a59e0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3f83348a 2021-08-11T16:20:16.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fc5aeb23-5a48-43c0-b138-2c47f36a59e0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:20:16.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fc5aeb23-5a48-43c0-b138-2c47f36a59e0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:20:18.309Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:20:18.317Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:20:19.759Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7cdcc542-44b4-4fa9-bb58-61a3b5c32a15 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:20:19.759Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7cdcc542-44b4-4fa9-bb58-61a3b5c32a15 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@692fba14 2021-08-11T16:20:19.759Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7cdcc542-44b4-4fa9-bb58-61a3b5c32a15 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:20:19.759Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7cdcc542-44b4-4fa9-bb58-61a3b5c32a15 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:20:22.637Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d148646d-406c-448b-ac8a-086aaa9aec9b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:20:22.637Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d148646d-406c-448b-ac8a-086aaa9aec9b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4f413a11 2021-08-11T16:20:22.637Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d148646d-406c-448b-ac8a-086aaa9aec9b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:20:22.637Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d148646d-406c-448b-ac8a-086aaa9aec9b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:20:25.751Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ea62293-0d4c-497a-a3be-3dd2a16cdc8c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:20:25.752Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ea62293-0d4c-497a-a3be-3dd2a16cdc8c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@361e22f8 2021-08-11T16:20:25.752Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ea62293-0d4c-497a-a3be-3dd2a16cdc8c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:20:25.752Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ea62293-0d4c-497a-a3be-3dd2a16cdc8c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:20:28.894Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2bcd0576-3ae6-419f-bc0a-38f5e5e7009a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:20:28.894Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2bcd0576-3ae6-419f-bc0a-38f5e5e7009a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@109f77d5 2021-08-11T16:20:28.894Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2bcd0576-3ae6-419f-bc0a-38f5e5e7009a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:20:28.894Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2bcd0576-3ae6-419f-bc0a-38f5e5e7009a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:20:31.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78a8efb0-1907-48f2-a89f-adef255b53d8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:20:31.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78a8efb0-1907-48f2-a89f-adef255b53d8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@47490c8 2021-08-11T16:20:31.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78a8efb0-1907-48f2-a89f-adef255b53d8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:20:31.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78a8efb0-1907-48f2-a89f-adef255b53d8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:20:33.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:20:33.318Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:20:34.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3d81971-f589-4c9d-a4c0-6b7ddb7bb9db PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:20:34.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3d81971-f589-4c9d-a4c0-6b7ddb7bb9db PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4223b3e5 2021-08-11T16:20:34.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3d81971-f589-4c9d-a4c0-6b7ddb7bb9db PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:20:34.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3d81971-f589-4c9d-a4c0-6b7ddb7bb9db PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:20:37.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4122d073-b7af-4912-ba81-e8a9fba96aee PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:20:37.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4122d073-b7af-4912-ba81-e8a9fba96aee PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@56d40bb 2021-08-11T16:20:37.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4122d073-b7af-4912-ba81-e8a9fba96aee PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:20:37.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4122d073-b7af-4912-ba81-e8a9fba96aee PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:20:40.634Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=acdd2cfa-969b-4ae2-9470-8bb0cc70b269 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:20:40.634Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=acdd2cfa-969b-4ae2-9470-8bb0cc70b269 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6ea7ae1c 2021-08-11T16:20:40.634Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=acdd2cfa-969b-4ae2-9470-8bb0cc70b269 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:20:40.634Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=acdd2cfa-969b-4ae2-9470-8bb0cc70b269 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:20:43.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b8ae64c6-adb1-4d86-9ff6-c497cc02ceaa PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:20:43.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b8ae64c6-adb1-4d86-9ff6-c497cc02ceaa PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4ef581d0 2021-08-11T16:20:43.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b8ae64c6-adb1-4d86-9ff6-c497cc02ceaa PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:20:43.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b8ae64c6-adb1-4d86-9ff6-c497cc02ceaa PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:20:46.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b173d35f-a838-4a02-9624-4bb526b58e8f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:20:46.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b173d35f-a838-4a02-9624-4bb526b58e8f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1196bf70 2021-08-11T16:20:46.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b173d35f-a838-4a02-9624-4bb526b58e8f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:20:46.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b173d35f-a838-4a02-9624-4bb526b58e8f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:20:48.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:20:48.311Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:20:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:20:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:20:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:20:50.356Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32cef6b0-e3cc-4bf6-92af-453d1676f78b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:20:50.356Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32cef6b0-e3cc-4bf6-92af-453d1676f78b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7e114d61 2021-08-11T16:20:50.356Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32cef6b0-e3cc-4bf6-92af-453d1676f78b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:20:50.356Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32cef6b0-e3cc-4bf6-92af-453d1676f78b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:20:53.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8bbe9e7b-c7d3-4945-96c5-53008f78495b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:20:53.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8bbe9e7b-c7d3-4945-96c5-53008f78495b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2d95cbf1 2021-08-11T16:20:53.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8bbe9e7b-c7d3-4945-96c5-53008f78495b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:20:53.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8bbe9e7b-c7d3-4945-96c5-53008f78495b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:20:56.745Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab4f44d0-14ed-4462-ac6d-1f1a0d905e5c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:20:56.745Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab4f44d0-14ed-4462-ac6d-1f1a0d905e5c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4f0c2492 2021-08-11T16:20:56.745Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab4f44d0-14ed-4462-ac6d-1f1a0d905e5c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:20:56.745Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab4f44d0-14ed-4462-ac6d-1f1a0d905e5c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:20:59.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c485cb85-4e9a-4c95-9291-3d8798e65cec PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:20:59.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c485cb85-4e9a-4c95-9291-3d8798e65cec PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6bc56f3c 2021-08-11T16:20:59.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c485cb85-4e9a-4c95-9291-3d8798e65cec PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:20:59.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c485cb85-4e9a-4c95-9291-3d8798e65cec PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:20:59.887Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=c485cb85-4e9a-4c95-9291-3d8798e65cec ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:20:59.887Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=c485cb85-4e9a-4c95-9291-3d8798e65cec ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:21:01.596Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02e4d8fb-5e11-4694-b2a2-7cd1554629c8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:21:01.596Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02e4d8fb-5e11-4694-b2a2-7cd1554629c8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1fcb5d60 2021-08-11T16:21:01.596Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02e4d8fb-5e11-4694-b2a2-7cd1554629c8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:21:01.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02e4d8fb-5e11-4694-b2a2-7cd1554629c8 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:21:01.611Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=02e4d8fb-5e11-4694-b2a2-7cd1554629c8 ErrorCategory=INFO ServiceName=BE Component CASSANDRA state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component CASSANDRA state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:21:01.611Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=02e4d8fb-5e11-4694-b2a2-7cd1554629c8 ErrorCategory=ERROR ServiceName=BE Component CASSANDRA state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-08-11T16:21:01.777Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:01.805Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:01.858Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:01.910Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:01.964Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:02.017Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:02.069Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:02.122Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:02.224Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:02.247Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:02.275Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:02.328Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:02.427Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:02.430Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:02.532Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:02.630Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:02.734Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:02.833Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:03.035Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:03.136Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:03.238Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:03.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:21:03.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-08-11T16:21:03.305Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:21:03.311Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:21:03.539Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:03.641Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:04.254Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:21:04.341Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:04.443Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:04.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=760c8ffb-69e9-4d32-a1a1-97a6293bf754 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:21:04.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=760c8ffb-69e9-4d32-a1a1-97a6293bf754 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1c02d72b 2021-08-11T16:21:04.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=760c8ffb-69e9-4d32-a1a1-97a6293bf754 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:21:04.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=760c8ffb-69e9-4d32-a1a1-97a6293bf754 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:21:04.604Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=760c8ffb-69e9-4d32-a1a1-97a6293bf754 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:21:04.604Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=760c8ffb-69e9-4d32-a1a1-97a6293bf754 ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-08-11T16:21:05.144Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:05.247Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T16:20:56.268333Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:06.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:21:06.748Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:07.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b18b209-e51a-4680-8fd0-715b55194826 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:21:07.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b18b209-e51a-4680-8fd0-715b55194826 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3284488d 2021-08-11T16:21:07.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b18b209-e51a-4680-8fd0-715b55194826 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:21:07.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b18b209-e51a-4680-8fd0-715b55194826 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:21:09.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:21:09.952Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:10.004Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:10.106Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:10.251Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:10.287Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:10.320Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:10.320Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:308) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:422) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:10.372Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:10.430Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:10.483Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:10.550Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:308) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:422) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:10.550Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:10.599Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44016b4d-1024-4e3d-9cf4-8a5691db70c5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:21:10.599Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44016b4d-1024-4e3d-9cf4-8a5691db70c5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@72786d6e 2021-08-11T16:21:10.599Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44016b4d-1024-4e3d-9cf4-8a5691db70c5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:21:10.600Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44016b4d-1024-4e3d-9cf4-8a5691db70c5 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:21:10.602Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:10.705Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:10.752Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:10.807Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:11.020Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:11.154Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:11.423Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:11.957Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:12.226Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:12.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:21:13.560Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:13.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58c11e1d-297a-4a52-919e-748d221fcad5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:21:13.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58c11e1d-297a-4a52-919e-748d221fcad5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@371d0e1c 2021-08-11T16:21:13.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58c11e1d-297a-4a52-919e-748d221fcad5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:21:13.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58c11e1d-297a-4a52-919e-748d221fcad5 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:21:13.828Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T16:20:56.268333Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:15.163Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:15.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:21:16.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=742362a0-ecaa-4ed1-a625-63db6504de01 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:21:16.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=742362a0-ecaa-4ed1-a625-63db6504de01 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3dac2c5b 2021-08-11T16:21:16.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=742362a0-ecaa-4ed1-a625-63db6504de01 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:21:16.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=742362a0-ecaa-4ed1-a625-63db6504de01 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:21:16.766Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:18.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:21:18.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:21:18.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:21:18.371Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:18.398Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:18.425Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:18.477Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:18.530Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:18.633Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:18.831Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:18.835Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:18.884Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:18.987Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:19.037Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:19.190Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:19.241Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:19.401Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:19.577Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=141a0e6a-545e-4cbd-b244-9a45932ecc4b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:21:19.577Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=141a0e6a-545e-4cbd-b244-9a45932ecc4b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2dfcdb43 2021-08-11T16:21:19.577Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=141a0e6a-545e-4cbd-b244-9a45932ecc4b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:21:19.577Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=141a0e6a-545e-4cbd-b244-9a45932ecc4b PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:21:19.604Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:19.644Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:19.806Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:20.209Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:20.446Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:20.611Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:21.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:21:21.415Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:22.049Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:22.218Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T16:20:56.268333Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:22.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f097855c-a120-4047-9048-596ab8472213 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:21:22.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f097855c-a120-4047-9048-596ab8472213 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1cded08f 2021-08-11T16:21:22.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f097855c-a120-4047-9048-596ab8472213 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:21:22.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f097855c-a120-4047-9048-596ab8472213 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:21:24.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:21:25.255Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:25.315Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:25.418Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:25.547Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:25.749Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87e039eb-f484-45a5-8f12-d5fae61449af PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:21:25.749Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87e039eb-f484-45a5-8f12-d5fae61449af PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@43ec50e6 2021-08-11T16:21:25.749Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87e039eb-f484-45a5-8f12-d5fae61449af PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:21:25.749Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:25.749Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87e039eb-f484-45a5-8f12-d5fae61449af PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:21:25.952Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:26.155Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:26.358Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:26.761Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:27.220Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:27.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:21:27.248Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:27.301Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:27.353Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:27.456Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:27.564Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:27.661Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:27.864Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:28.267Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:28.670Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:28.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0c782c26-b6a3-4fcc-8726-4777697eac22 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:21:28.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0c782c26-b6a3-4fcc-8726-4777697eac22 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1ffe6234 2021-08-11T16:21:28.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0c782c26-b6a3-4fcc-8726-4777697eac22 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:21:28.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0c782c26-b6a3-4fcc-8726-4777697eac22 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:21:29.166Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:29.473Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:30.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:21:30.276Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:30.768Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:31.078Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T16:20:56.268333Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:31.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=af8d48c8-25a5-48fb-9364-15378df31322 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:21:31.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=af8d48c8-25a5-48fb-9364-15378df31322 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5413944b 2021-08-11T16:21:31.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=af8d48c8-25a5-48fb-9364-15378df31322 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:21:31.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=af8d48c8-25a5-48fb-9364-15378df31322 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:21:32.373Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:32.426Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:32.478Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:32.580Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:32.783Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:32.986Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:33.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:21:33.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:21:33.317Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:21:33.389Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:33.792Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:34.577Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a61c55d-ff74-4783-bf11-766c590da425 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:21:34.577Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a61c55d-ff74-4783-bf11-766c590da425 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3ea36cee 2021-08-11T16:21:34.577Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a61c55d-ff74-4783-bf11-766c590da425 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:21:34.577Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7a61c55d-ff74-4783-bf11-766c590da425 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:21:34.595Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:35.397Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:36.081Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:36.109Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:36.137Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:36.164Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:36.191Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:36.200Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:36.244Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:36.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:21:36.347Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:36.449Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:36.552Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:36.655Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:36.858Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:37.002Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:37.261Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:37.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8d34448-dc16-4d93-9e35-439df2604cf8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:21:37.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8d34448-dc16-4d93-9e35-439df2604cf8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1fe970bc 2021-08-11T16:21:37.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8d34448-dc16-4d93-9e35-439df2604cf8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:21:37.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8d34448-dc16-4d93-9e35-439df2604cf8 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:21:37.805Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:38.064Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:38.867Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T16:20:56.268333Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:39.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:21:39.448Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:41.047Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=39a1a6c4-61a4-4eb4-bbf5-d6f84393d53a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:21:41.047Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=39a1a6c4-61a4-4eb4-bbf5-d6f84393d53a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@736a4eb7 2021-08-11T16:21:41.047Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=39a1a6c4-61a4-4eb4-bbf5-d6f84393d53a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:21:41.047Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=39a1a6c4-61a4-4eb4-bbf5-d6f84393d53a PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:21:41.051Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:41.148Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:41.201Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:41.304Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:41.507Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:41.911Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:42.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:21:42.314Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:42.717Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:43.521Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:43.595Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5669bd03-3992-4b0c-b759-9c740464cefa PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:21:43.595Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5669bd03-3992-4b0c-b759-9c740464cefa PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3b132d5d 2021-08-11T16:21:43.595Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5669bd03-3992-4b0c-b759-9c740464cefa PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:21:43.595Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5669bd03-3992-4b0c-b759-9c740464cefa PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:21:43.869Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:43.921Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:43.974Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:44.027Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:44.131Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:44.234Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:44.325Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:44.437Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:21:45.245Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:21:46.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b9d973dc-284d-43dd-b80e-50db488d2036 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:21:46.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b9d973dc-284d-43dd-b80e-50db488d2036 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1e72f2cd 2021-08-11T16:21:46.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b9d973dc-284d-43dd-b80e-50db488d2036 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:21:46.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b9d973dc-284d-43dd-b80e-50db488d2036 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:21:47.264Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-08-11T16:21:48.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:21:48.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:21:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:21:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:21:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:21:49.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95319379-15e8-4cd7-80f9-ef3409e757db PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:21:49.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95319379-15e8-4cd7-80f9-ef3409e757db PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4aca1d2f 2021-08-11T16:21:49.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95319379-15e8-4cd7-80f9-ef3409e757db PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:21:49.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95319379-15e8-4cd7-80f9-ef3409e757db PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:21:49.971Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=95319379-15e8-4cd7-80f9-ef3409e757db ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:21:49.971Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=95319379-15e8-4cd7-80f9-ef3409e757db ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-08-11T16:21:52.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e66259a5-7194-49a9-85d4-ecc345228823 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:21:52.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e66259a5-7194-49a9-85d4-ecc345228823 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6b74fd37 2021-08-11T16:21:52.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e66259a5-7194-49a9-85d4-ecc345228823 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:21:52.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e66259a5-7194-49a9-85d4-ecc345228823 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:21:55.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18e4d308-95b5-4cc0-b996-6aeec574e4d1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:21:55.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18e4d308-95b5-4cc0-b996-6aeec574e4d1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@49af5cc1 2021-08-11T16:21:55.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18e4d308-95b5-4cc0-b996-6aeec574e4d1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:21:55.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18e4d308-95b5-4cc0-b996-6aeec574e4d1 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:22:00.346Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=811488f4-ee84-4763-9a2f-267f4523624f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:22:00.346Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=811488f4-ee84-4763-9a2f-267f4523624f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@40422ba3 2021-08-11T16:22:00.346Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=811488f4-ee84-4763-9a2f-267f4523624f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:22:00.346Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=811488f4-ee84-4763-9a2f-267f4523624f PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:22:00.777Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=811488f4-ee84-4763-9a2f-267f4523624f ErrorCategory=INFO ServiceName=BE Component CASSANDRA state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component CASSANDRA state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:22:00.777Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=811488f4-ee84-4763-9a2f-267f4523624f ErrorCategory=INFO ServiceName=BE Component CASSANDRA state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:22:03.310Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:22:03.317Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:22:04.308Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:22:04.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a007354-26db-47f1-bcd6-afeaeb2295e3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:22:04.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a007354-26db-47f1-bcd6-afeaeb2295e3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5a7b65e1 2021-08-11T16:22:04.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a007354-26db-47f1-bcd6-afeaeb2295e3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:22:04.569Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a007354-26db-47f1-bcd6-afeaeb2295e3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:22:07.387Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c9cfc16-a3c6-4f8f-bd1e-3705942147e9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:22:07.387Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c9cfc16-a3c6-4f8f-bd1e-3705942147e9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@16aab6dc 2021-08-11T16:22:07.387Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c9cfc16-a3c6-4f8f-bd1e-3705942147e9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:22:07.388Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c9cfc16-a3c6-4f8f-bd1e-3705942147e9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:22:08.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86a7b5d0-8da9-40ec-a286-33e570949b08 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:22:08.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86a7b5d0-8da9-40ec-a286-33e570949b08 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@25d0a99c 2021-08-11T16:22:08.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86a7b5d0-8da9-40ec-a286-33e570949b08 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:22:08.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86a7b5d0-8da9-40ec-a286-33e570949b08 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:22:10.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=084b53e7-593b-40b5-9977-771cd846062f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:22:10.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=084b53e7-593b-40b5-9977-771cd846062f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@749e81b6 2021-08-11T16:22:10.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=084b53e7-593b-40b5-9977-771cd846062f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:22:10.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=084b53e7-593b-40b5-9977-771cd846062f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:22:14.365Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=722ec341-2b32-4a91-b9f2-f1d2a41b3033 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:22:14.365Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=722ec341-2b32-4a91-b9f2-f1d2a41b3033 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@754bb6a9 2021-08-11T16:22:14.365Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=722ec341-2b32-4a91-b9f2-f1d2a41b3033 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:22:14.366Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=722ec341-2b32-4a91-b9f2-f1d2a41b3033 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:22:16.847Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba5cf2ac-768f-428e-ba93-7e0bc88d4087 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:22:16.847Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba5cf2ac-768f-428e-ba93-7e0bc88d4087 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3614e28a 2021-08-11T16:22:16.847Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba5cf2ac-768f-428e-ba93-7e0bc88d4087 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:22:16.847Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba5cf2ac-768f-428e-ba93-7e0bc88d4087 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:22:18.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:22:18.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:22:19.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6b110359-8b7a-4e92-a07d-12aecdefed1c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:22:19.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6b110359-8b7a-4e92-a07d-12aecdefed1c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1628219d 2021-08-11T16:22:19.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6b110359-8b7a-4e92-a07d-12aecdefed1c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:22:19.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6b110359-8b7a-4e92-a07d-12aecdefed1c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:22:22.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aef18a00-745b-4c2a-85da-07b6a3fec421 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:22:22.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aef18a00-745b-4c2a-85da-07b6a3fec421 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@163b81df 2021-08-11T16:22:22.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aef18a00-745b-4c2a-85da-07b6a3fec421 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:22:22.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aef18a00-745b-4c2a-85da-07b6a3fec421 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:22:25.688Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6185443-7fde-4da8-94e7-beca66edc76b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:22:25.689Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6185443-7fde-4da8-94e7-beca66edc76b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3e6d934f 2021-08-11T16:22:25.689Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6185443-7fde-4da8-94e7-beca66edc76b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:22:25.689Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6185443-7fde-4da8-94e7-beca66edc76b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:22:28.987Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78475e4d-78e5-4567-85a6-0ce4ea9a615f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:22:28.987Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78475e4d-78e5-4567-85a6-0ce4ea9a615f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2f7c26ba 2021-08-11T16:22:28.988Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78475e4d-78e5-4567-85a6-0ce4ea9a615f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:22:28.988Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=78475e4d-78e5-4567-85a6-0ce4ea9a615f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:22:31.932Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3386a9f7-73e7-4310-a27f-82ffc1904795 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:22:31.932Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3386a9f7-73e7-4310-a27f-82ffc1904795 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@780fce01 2021-08-11T16:22:31.932Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3386a9f7-73e7-4310-a27f-82ffc1904795 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:22:31.932Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3386a9f7-73e7-4310-a27f-82ffc1904795 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:22:33.305Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:22:33.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:22:34.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a9732e3-c43b-4443-8f5c-eb7618a3c743 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:22:34.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a9732e3-c43b-4443-8f5c-eb7618a3c743 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@32708da6 2021-08-11T16:22:34.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a9732e3-c43b-4443-8f5c-eb7618a3c743 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:22:34.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a9732e3-c43b-4443-8f5c-eb7618a3c743 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:22:37.975Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8c4d8f04-1a1d-42c1-800a-519bffe4b82f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:22:37.975Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8c4d8f04-1a1d-42c1-800a-519bffe4b82f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@60d5974c 2021-08-11T16:22:37.975Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8c4d8f04-1a1d-42c1-800a-519bffe4b82f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:22:37.975Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8c4d8f04-1a1d-42c1-800a-519bffe4b82f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:22:40.694Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2d1b9d04-11dc-4792-ac40-1cd93bf4b483 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:22:40.695Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2d1b9d04-11dc-4792-ac40-1cd93bf4b483 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2e52241a 2021-08-11T16:22:40.695Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2d1b9d04-11dc-4792-ac40-1cd93bf4b483 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:22:40.695Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2d1b9d04-11dc-4792-ac40-1cd93bf4b483 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:22:43.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64ecdb0b-ee5b-4574-b1fd-9ca5dedbddf9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:22:43.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64ecdb0b-ee5b-4574-b1fd-9ca5dedbddf9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@282b8817 2021-08-11T16:22:43.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64ecdb0b-ee5b-4574-b1fd-9ca5dedbddf9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:22:43.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64ecdb0b-ee5b-4574-b1fd-9ca5dedbddf9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:22:46.699Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f87e39fb-6b55-49b6-92c7-f3290cd9452f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:22:46.699Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f87e39fb-6b55-49b6-92c7-f3290cd9452f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1e803d09 2021-08-11T16:22:46.699Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f87e39fb-6b55-49b6-92c7-f3290cd9452f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:22:46.699Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f87e39fb-6b55-49b6-92c7-f3290cd9452f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:22:48.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:22:48.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:22:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:22:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:22:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:22:49.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1fab86cf-0f7b-48a6-8383-99114b2aea03 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:22:49.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1fab86cf-0f7b-48a6-8383-99114b2aea03 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5223a662 2021-08-11T16:22:49.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1fab86cf-0f7b-48a6-8383-99114b2aea03 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:22:49.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1fab86cf-0f7b-48a6-8383-99114b2aea03 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:22:52.736Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1dfd0907-74ce-4a1d-98e1-1caca3203e83 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:22:52.736Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1dfd0907-74ce-4a1d-98e1-1caca3203e83 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6d6e3ac8 2021-08-11T16:22:52.736Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1dfd0907-74ce-4a1d-98e1-1caca3203e83 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:22:52.736Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1dfd0907-74ce-4a1d-98e1-1caca3203e83 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:22:55.960Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d76970a4-deb2-4603-96f7-43cc15b1d2bf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:22:55.960Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d76970a4-deb2-4603-96f7-43cc15b1d2bf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@19f17c42 2021-08-11T16:22:55.960Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d76970a4-deb2-4603-96f7-43cc15b1d2bf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:22:55.960Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d76970a4-deb2-4603-96f7-43cc15b1d2bf PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:22:58.694Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=91aae4e1-3160-41e1-881d-aec2b78c91bd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:22:58.694Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=91aae4e1-3160-41e1-881d-aec2b78c91bd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6866200 2021-08-11T16:22:58.694Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=91aae4e1-3160-41e1-881d-aec2b78c91bd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:22:58.694Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=91aae4e1-3160-41e1-881d-aec2b78c91bd PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:23:01.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=989fc452-8369-4272-82f0-85ecb2339e22 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:23:01.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=989fc452-8369-4272-82f0-85ecb2339e22 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4959c404 2021-08-11T16:23:01.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=989fc452-8369-4272-82f0-85ecb2339e22 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:23:01.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=989fc452-8369-4272-82f0-85ecb2339e22 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:23:03.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:23:03.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:23:04.256Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:23:04.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2bbfd780-b61a-4f6d-936c-777ef50b178e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:23:04.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2bbfd780-b61a-4f6d-936c-777ef50b178e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@557d1f9d 2021-08-11T16:23:04.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2bbfd780-b61a-4f6d-936c-777ef50b178e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:23:04.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2bbfd780-b61a-4f6d-936c-777ef50b178e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:23:07.628Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed3e50ba-0e2a-4025-8655-032ad9cb1770 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:23:07.628Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed3e50ba-0e2a-4025-8655-032ad9cb1770 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@47fc6a1c 2021-08-11T16:23:07.628Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed3e50ba-0e2a-4025-8655-032ad9cb1770 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:23:07.628Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed3e50ba-0e2a-4025-8655-032ad9cb1770 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:23:10.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ee1cc4d-a252-4392-abfa-2be3c20dec6f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:23:10.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ee1cc4d-a252-4392-abfa-2be3c20dec6f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@725f8681 2021-08-11T16:23:10.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ee1cc4d-a252-4392-abfa-2be3c20dec6f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:23:10.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ee1cc4d-a252-4392-abfa-2be3c20dec6f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:23:13.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=62745bed-0237-41f9-92ea-93ff5528e034 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:23:13.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=62745bed-0237-41f9-92ea-93ff5528e034 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7521052a 2021-08-11T16:23:13.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=62745bed-0237-41f9-92ea-93ff5528e034 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:23:13.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=62745bed-0237-41f9-92ea-93ff5528e034 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:23:16.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a20b433-6f92-4d9d-a992-7dff8548c039 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:23:16.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a20b433-6f92-4d9d-a992-7dff8548c039 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@481d6de4 2021-08-11T16:23:16.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a20b433-6f92-4d9d-a992-7dff8548c039 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:23:16.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a20b433-6f92-4d9d-a992-7dff8548c039 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:23:18.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:23:18.319Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:23:19.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa43552f-86ff-4237-a397-08c513014b84 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:23:19.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa43552f-86ff-4237-a397-08c513014b84 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6c954271 2021-08-11T16:23:19.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa43552f-86ff-4237-a397-08c513014b84 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:23:19.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa43552f-86ff-4237-a397-08c513014b84 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:23:22.612Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2482dfa8-1a72-48a7-89c7-e343db532867 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:23:22.612Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2482dfa8-1a72-48a7-89c7-e343db532867 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@62f0aaec 2021-08-11T16:23:22.612Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2482dfa8-1a72-48a7-89c7-e343db532867 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:23:22.612Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2482dfa8-1a72-48a7-89c7-e343db532867 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:23:25.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b38371cc-c1ab-48a5-990e-f7baf45170c0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:23:25.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b38371cc-c1ab-48a5-990e-f7baf45170c0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1f05fbcd 2021-08-11T16:23:25.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b38371cc-c1ab-48a5-990e-f7baf45170c0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:23:25.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b38371cc-c1ab-48a5-990e-f7baf45170c0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:23:28.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ef2b3d2-280e-404b-8861-2f935b739859 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:23:28.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ef2b3d2-280e-404b-8861-2f935b739859 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3cd4d784 2021-08-11T16:23:28.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ef2b3d2-280e-404b-8861-2f935b739859 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:23:28.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ef2b3d2-280e-404b-8861-2f935b739859 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:23:31.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ffbad97f-6f33-4c6b-a04a-8de4a97a3257 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:23:31.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ffbad97f-6f33-4c6b-a04a-8de4a97a3257 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7a949ca5 2021-08-11T16:23:31.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ffbad97f-6f33-4c6b-a04a-8de4a97a3257 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:23:31.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ffbad97f-6f33-4c6b-a04a-8de4a97a3257 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:23:33.309Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:23:33.317Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:23:34.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ada95035-8b8c-499b-8279-ac49f176066e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:23:34.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ada95035-8b8c-499b-8279-ac49f176066e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1dd4eed1 2021-08-11T16:23:34.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ada95035-8b8c-499b-8279-ac49f176066e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:23:34.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ada95035-8b8c-499b-8279-ac49f176066e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:23:37.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c25fd07c-426a-4287-8a96-555c61a01dc7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:23:37.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c25fd07c-426a-4287-8a96-555c61a01dc7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2dbfa535 2021-08-11T16:23:37.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c25fd07c-426a-4287-8a96-555c61a01dc7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:23:37.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c25fd07c-426a-4287-8a96-555c61a01dc7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:23:40.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1228278c-3c32-4e13-8a52-68ff42602ef9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:23:40.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1228278c-3c32-4e13-8a52-68ff42602ef9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@374d2ff1 2021-08-11T16:23:40.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1228278c-3c32-4e13-8a52-68ff42602ef9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:23:40.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1228278c-3c32-4e13-8a52-68ff42602ef9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:23:43.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc454ed8-b7e1-4a25-bd47-e6d6568261ce PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:23:43.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc454ed8-b7e1-4a25-bd47-e6d6568261ce PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6962ec09 2021-08-11T16:23:43.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc454ed8-b7e1-4a25-bd47-e6d6568261ce PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:23:43.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc454ed8-b7e1-4a25-bd47-e6d6568261ce PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:23:46.627Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a895ef4a-ac52-4fde-b3c2-cabc092b909f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:23:46.627Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a895ef4a-ac52-4fde-b3c2-cabc092b909f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@537a2592 2021-08-11T16:23:46.627Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a895ef4a-ac52-4fde-b3c2-cabc092b909f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:23:46.627Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a895ef4a-ac52-4fde-b3c2-cabc092b909f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:23:48.309Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:23:48.317Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:23:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:23:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:23:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:23:50.059Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a933eb4-39e4-4117-bc25-64d97ad5d2af PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:23:50.060Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a933eb4-39e4-4117-bc25-64d97ad5d2af PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3bca4c5 2021-08-11T16:23:50.060Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a933eb4-39e4-4117-bc25-64d97ad5d2af PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:23:50.060Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a933eb4-39e4-4117-bc25-64d97ad5d2af PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:23:53.378Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1e48a85-1116-4e08-b4c6-912db8d39d8d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:23:53.378Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1e48a85-1116-4e08-b4c6-912db8d39d8d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5dd021e4 2021-08-11T16:23:53.378Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1e48a85-1116-4e08-b4c6-912db8d39d8d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:23:53.378Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1e48a85-1116-4e08-b4c6-912db8d39d8d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:23:56.273Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=67a15bb7-68ee-4054-a586-094b7ec2e915 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:23:56.273Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=67a15bb7-68ee-4054-a586-094b7ec2e915 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@23ac9cf3 2021-08-11T16:23:56.273Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=67a15bb7-68ee-4054-a586-094b7ec2e915 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:23:56.273Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=67a15bb7-68ee-4054-a586-094b7ec2e915 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:23:58.967Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2a6e260-2592-4a1a-9e12-23c1ee8842bc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:23:58.967Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2a6e260-2592-4a1a-9e12-23c1ee8842bc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@21f86ba1 2021-08-11T16:23:58.967Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2a6e260-2592-4a1a-9e12-23c1ee8842bc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:23:58.968Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2a6e260-2592-4a1a-9e12-23c1ee8842bc PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:24:01.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a42d1b6-bfe9-40d6-8080-2ec904dc8e3d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:24:01.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a42d1b6-bfe9-40d6-8080-2ec904dc8e3d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@470469d7 2021-08-11T16:24:01.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a42d1b6-bfe9-40d6-8080-2ec904dc8e3d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:24:01.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a42d1b6-bfe9-40d6-8080-2ec904dc8e3d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:24:03.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:24:03.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:24:04.260Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:24:04.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b551b44-e4d4-4b97-97cd-9da21c566fad PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:24:04.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b551b44-e4d4-4b97-97cd-9da21c566fad PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1b87afd2 2021-08-11T16:24:04.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b551b44-e4d4-4b97-97cd-9da21c566fad PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:24:04.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b551b44-e4d4-4b97-97cd-9da21c566fad PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:24:07.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9b35de36-223c-46f2-8f20-93e8ff16eb68 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:24:07.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9b35de36-223c-46f2-8f20-93e8ff16eb68 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5c0bf355 2021-08-11T16:24:07.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9b35de36-223c-46f2-8f20-93e8ff16eb68 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:24:07.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9b35de36-223c-46f2-8f20-93e8ff16eb68 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:24:10.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad0d9b72-3458-45a3-b14a-5e7f239a4e42 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:24:10.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad0d9b72-3458-45a3-b14a-5e7f239a4e42 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3ecc3e16 2021-08-11T16:24:10.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad0d9b72-3458-45a3-b14a-5e7f239a4e42 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:24:10.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad0d9b72-3458-45a3-b14a-5e7f239a4e42 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:24:13.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24c0c837-2c62-445d-8d9f-d33458026754 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:24:13.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24c0c837-2c62-445d-8d9f-d33458026754 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5a62a2c9 2021-08-11T16:24:13.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24c0c837-2c62-445d-8d9f-d33458026754 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:24:13.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24c0c837-2c62-445d-8d9f-d33458026754 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:24:16.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c8e3e52-5bf4-4fd5-81c8-81e30d42e1bd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:24:16.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c8e3e52-5bf4-4fd5-81c8-81e30d42e1bd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7c3d7f68 2021-08-11T16:24:16.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c8e3e52-5bf4-4fd5-81c8-81e30d42e1bd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:24:16.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c8e3e52-5bf4-4fd5-81c8-81e30d42e1bd PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:24:18.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:24:18.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:24:19.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58ed707b-3222-4a5e-9734-b18e075cb6bb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:24:19.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58ed707b-3222-4a5e-9734-b18e075cb6bb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7fafbae1 2021-08-11T16:24:19.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58ed707b-3222-4a5e-9734-b18e075cb6bb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:24:19.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58ed707b-3222-4a5e-9734-b18e075cb6bb PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:24:22.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d1d5110-82b9-4ead-9c5e-227c31f31b28 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:24:22.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d1d5110-82b9-4ead-9c5e-227c31f31b28 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@20cc078c 2021-08-11T16:24:22.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d1d5110-82b9-4ead-9c5e-227c31f31b28 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:24:22.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d1d5110-82b9-4ead-9c5e-227c31f31b28 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:24:25.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b14ef4bb-1d7f-4a03-9d0d-04e6538eb5cf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:24:25.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b14ef4bb-1d7f-4a03-9d0d-04e6538eb5cf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6f779db6 2021-08-11T16:24:25.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b14ef4bb-1d7f-4a03-9d0d-04e6538eb5cf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:24:25.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b14ef4bb-1d7f-4a03-9d0d-04e6538eb5cf PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:24:28.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a7380259-2e24-4ec7-8379-abb7e1355c62 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:24:28.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a7380259-2e24-4ec7-8379-abb7e1355c62 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7826790f 2021-08-11T16:24:28.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a7380259-2e24-4ec7-8379-abb7e1355c62 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:24:28.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a7380259-2e24-4ec7-8379-abb7e1355c62 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:24:31.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9477da16-389d-47a9-add0-3ef60f6ca5bf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:24:31.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9477da16-389d-47a9-add0-3ef60f6ca5bf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7a5cbc58 2021-08-11T16:24:31.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9477da16-389d-47a9-add0-3ef60f6ca5bf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:24:31.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9477da16-389d-47a9-add0-3ef60f6ca5bf PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:24:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:24:33.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:24:34.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae36537f-82d0-4d6f-8624-00cd080cb915 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:24:34.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae36537f-82d0-4d6f-8624-00cd080cb915 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3aec24f8 2021-08-11T16:24:34.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae36537f-82d0-4d6f-8624-00cd080cb915 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:24:34.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae36537f-82d0-4d6f-8624-00cd080cb915 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:24:37.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b6353ea1-c8d8-47b6-a53a-1ed6ab1b617e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:24:37.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b6353ea1-c8d8-47b6-a53a-1ed6ab1b617e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6925943e 2021-08-11T16:24:37.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b6353ea1-c8d8-47b6-a53a-1ed6ab1b617e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:24:37.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b6353ea1-c8d8-47b6-a53a-1ed6ab1b617e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:24:40.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5f99f72-45eb-40e9-869d-c3bd19831f68 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:24:40.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5f99f72-45eb-40e9-869d-c3bd19831f68 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@22807614 2021-08-11T16:24:40.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5f99f72-45eb-40e9-869d-c3bd19831f68 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:24:40.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5f99f72-45eb-40e9-869d-c3bd19831f68 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:24:43.640Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec63c9b2-ded4-42de-a60f-3a2f2e08b578 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:24:43.640Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec63c9b2-ded4-42de-a60f-3a2f2e08b578 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3da1b060 2021-08-11T16:24:43.640Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec63c9b2-ded4-42de-a60f-3a2f2e08b578 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:24:43.640Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec63c9b2-ded4-42de-a60f-3a2f2e08b578 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:24:46.713Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d763746d-d552-4dc4-8791-9e7b6f89c9a5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:24:46.713Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d763746d-d552-4dc4-8791-9e7b6f89c9a5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@663188c8 2021-08-11T16:24:46.713Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d763746d-d552-4dc4-8791-9e7b6f89c9a5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:24:46.713Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d763746d-d552-4dc4-8791-9e7b6f89c9a5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:24:48.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:24:48.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:24:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:24:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:24:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:24:50.158Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b93efdf1-e1eb-40fd-a95c-9673fd8545e1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:24:50.158Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b93efdf1-e1eb-40fd-a95c-9673fd8545e1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@23f07454 2021-08-11T16:24:50.158Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b93efdf1-e1eb-40fd-a95c-9673fd8545e1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:24:50.158Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b93efdf1-e1eb-40fd-a95c-9673fd8545e1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:24:53.472Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c5f65981-f81b-4673-bb83-140985b3a661 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:24:53.473Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c5f65981-f81b-4673-bb83-140985b3a661 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2e186e8 2021-08-11T16:24:53.473Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c5f65981-f81b-4673-bb83-140985b3a661 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:24:53.473Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c5f65981-f81b-4673-bb83-140985b3a661 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:24:56.961Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=67713f51-283d-46bf-93cf-1149fbd9f758 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:24:56.961Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=67713f51-283d-46bf-93cf-1149fbd9f758 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@32730464 2021-08-11T16:24:56.961Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=67713f51-283d-46bf-93cf-1149fbd9f758 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:24:56.961Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=67713f51-283d-46bf-93cf-1149fbd9f758 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:24:59.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c2b1ac0a-0c5a-46f1-9d94-bc6d90c39113 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:24:59.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c2b1ac0a-0c5a-46f1-9d94-bc6d90c39113 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@586ebd2a 2021-08-11T16:24:59.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c2b1ac0a-0c5a-46f1-9d94-bc6d90c39113 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:24:59.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c2b1ac0a-0c5a-46f1-9d94-bc6d90c39113 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:25:03.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:25:03.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:25:03.362Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9042890d-66c3-4fc5-b0c5-84c000888f1b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:25:03.362Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9042890d-66c3-4fc5-b0c5-84c000888f1b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@781c30a7 2021-08-11T16:25:03.362Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9042890d-66c3-4fc5-b0c5-84c000888f1b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:25:03.362Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9042890d-66c3-4fc5-b0c5-84c000888f1b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:25:04.253Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:25:05.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f132da6f-90d7-4cad-b41a-b4cee73860a7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:25:05.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f132da6f-90d7-4cad-b41a-b4cee73860a7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@76442f8a 2021-08-11T16:25:05.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f132da6f-90d7-4cad-b41a-b4cee73860a7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:25:05.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f132da6f-90d7-4cad-b41a-b4cee73860a7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:25:08.062Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=960111f5-d4fe-4751-bcd4-fc3bc9559f51 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:25:08.062Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=960111f5-d4fe-4751-bcd4-fc3bc9559f51 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5cbe4502 2021-08-11T16:25:08.062Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=960111f5-d4fe-4751-bcd4-fc3bc9559f51 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:25:08.062Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=960111f5-d4fe-4751-bcd4-fc3bc9559f51 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:25:10.594Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd36dfe2-71a9-47e8-a736-55740df5fbc6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:25:10.594Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd36dfe2-71a9-47e8-a736-55740df5fbc6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@71142452 2021-08-11T16:25:10.594Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd36dfe2-71a9-47e8-a736-55740df5fbc6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:25:10.594Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd36dfe2-71a9-47e8-a736-55740df5fbc6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:25:13.658Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21a2cc52-f337-4b64-a0fb-3ce468cfcae2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:25:13.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21a2cc52-f337-4b64-a0fb-3ce468cfcae2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6814bbdf 2021-08-11T16:25:13.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21a2cc52-f337-4b64-a0fb-3ce468cfcae2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:25:13.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21a2cc52-f337-4b64-a0fb-3ce468cfcae2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:25:16.696Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73db4fba-54d6-47f6-9b7d-8af211b580c3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:25:16.696Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73db4fba-54d6-47f6-9b7d-8af211b580c3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3ffbe128 2021-08-11T16:25:16.696Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73db4fba-54d6-47f6-9b7d-8af211b580c3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:25:16.696Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73db4fba-54d6-47f6-9b7d-8af211b580c3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:25:18.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:25:18.319Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:25:19.628Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ff8638d1-f97a-426f-b3a4-cb3c086753ad PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:25:19.628Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ff8638d1-f97a-426f-b3a4-cb3c086753ad PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@11c32580 2021-08-11T16:25:19.628Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ff8638d1-f97a-426f-b3a4-cb3c086753ad PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:25:19.628Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ff8638d1-f97a-426f-b3a4-cb3c086753ad PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:25:22.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5120629-c6c5-493f-9b53-1cc500e1fed0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:25:22.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5120629-c6c5-493f-9b53-1cc500e1fed0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4125b55e 2021-08-11T16:25:22.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5120629-c6c5-493f-9b53-1cc500e1fed0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:25:22.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5120629-c6c5-493f-9b53-1cc500e1fed0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:25:25.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ebf72b9d-7215-4b2c-a376-ecb3454ab15d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:25:25.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ebf72b9d-7215-4b2c-a376-ecb3454ab15d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@64d7b1c8 2021-08-11T16:25:25.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ebf72b9d-7215-4b2c-a376-ecb3454ab15d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:25:25.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ebf72b9d-7215-4b2c-a376-ecb3454ab15d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:25:28.853Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d262d738-ce6c-49bb-9b4f-3dbb315d2626 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:25:28.853Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d262d738-ce6c-49bb-9b4f-3dbb315d2626 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@8347749 2021-08-11T16:25:28.853Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d262d738-ce6c-49bb-9b4f-3dbb315d2626 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:25:28.853Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d262d738-ce6c-49bb-9b4f-3dbb315d2626 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:25:31.751Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8872fd4b-b838-4e89-9a54-3631c73d376e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:25:31.752Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8872fd4b-b838-4e89-9a54-3631c73d376e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@152c77ad 2021-08-11T16:25:31.752Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8872fd4b-b838-4e89-9a54-3631c73d376e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:25:31.752Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8872fd4b-b838-4e89-9a54-3631c73d376e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:25:33.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:25:33.322Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:25:34.636Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=adf0a5d0-6972-46ab-ac03-b747d79f74fa PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:25:34.636Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=adf0a5d0-6972-46ab-ac03-b747d79f74fa PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7ac44c4d 2021-08-11T16:25:34.636Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=adf0a5d0-6972-46ab-ac03-b747d79f74fa PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:25:34.637Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=adf0a5d0-6972-46ab-ac03-b747d79f74fa PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:25:37.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50baab8e-0796-4005-9df8-aa0fce4a20bd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:25:37.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50baab8e-0796-4005-9df8-aa0fce4a20bd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2344eb2c 2021-08-11T16:25:37.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50baab8e-0796-4005-9df8-aa0fce4a20bd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:25:37.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50baab8e-0796-4005-9df8-aa0fce4a20bd PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:25:40.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=304f43c2-2851-419a-a87a-6f473ba6973f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:25:40.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=304f43c2-2851-419a-a87a-6f473ba6973f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@8614e32 2021-08-11T16:25:40.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=304f43c2-2851-419a-a87a-6f473ba6973f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:25:40.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=304f43c2-2851-419a-a87a-6f473ba6973f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:25:43.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0b296cb7-900f-4b1c-89c4-137bac79509a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:25:43.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0b296cb7-900f-4b1c-89c4-137bac79509a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7e86c18c 2021-08-11T16:25:43.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0b296cb7-900f-4b1c-89c4-137bac79509a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:25:43.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0b296cb7-900f-4b1c-89c4-137bac79509a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:25:46.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c3e4fce-7717-483b-9adf-8b586837166f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:25:46.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c3e4fce-7717-483b-9adf-8b586837166f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@26784c35 2021-08-11T16:25:46.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c3e4fce-7717-483b-9adf-8b586837166f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:25:46.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c3e4fce-7717-483b-9adf-8b586837166f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:25:48.309Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:25:48.316Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:25:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:25:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:25:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:25:50.349Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f966946c-bc13-4751-910e-85063035f28d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:25:50.349Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f966946c-bc13-4751-910e-85063035f28d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@22e1de5d 2021-08-11T16:25:50.349Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f966946c-bc13-4751-910e-85063035f28d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:25:50.349Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f966946c-bc13-4751-910e-85063035f28d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:25:53.275Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b68a17e7-8e19-4d64-af49-7f6d8c9b61b5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:25:53.275Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b68a17e7-8e19-4d64-af49-7f6d8c9b61b5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@20f304ed 2021-08-11T16:25:53.275Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b68a17e7-8e19-4d64-af49-7f6d8c9b61b5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:25:53.275Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b68a17e7-8e19-4d64-af49-7f6d8c9b61b5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:25:56.476Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a15590a-d164-4a07-8be0-8c710a94e1a5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:25:56.476Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a15590a-d164-4a07-8be0-8c710a94e1a5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@68313bde 2021-08-11T16:25:56.476Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a15590a-d164-4a07-8be0-8c710a94e1a5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:25:56.476Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a15590a-d164-4a07-8be0-8c710a94e1a5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:25:59.362Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a7cb30e-9af8-4f70-acfc-2788a1a748c6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:25:59.362Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a7cb30e-9af8-4f70-acfc-2788a1a748c6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@cccdb09 2021-08-11T16:25:59.362Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a7cb30e-9af8-4f70-acfc-2788a1a748c6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:25:59.362Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a7cb30e-9af8-4f70-acfc-2788a1a748c6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:26:01.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37ab12b0-7ddd-41f7-a40b-07f6893a8247 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:26:01.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37ab12b0-7ddd-41f7-a40b-07f6893a8247 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2073fffb 2021-08-11T16:26:01.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37ab12b0-7ddd-41f7-a40b-07f6893a8247 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:26:01.654Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37ab12b0-7ddd-41f7-a40b-07f6893a8247 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:26:03.309Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:26:03.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:26:04.330Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:26:04.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e3a2754b-8827-4e04-aaad-1d6ee429c8c5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:26:04.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e3a2754b-8827-4e04-aaad-1d6ee429c8c5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@189f5374 2021-08-11T16:26:04.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e3a2754b-8827-4e04-aaad-1d6ee429c8c5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:26:04.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e3a2754b-8827-4e04-aaad-1d6ee429c8c5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:26:07.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f9352c6f-8d9c-446d-a826-3b0a73cc749a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:26:07.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f9352c6f-8d9c-446d-a826-3b0a73cc749a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@79116472 2021-08-11T16:26:07.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f9352c6f-8d9c-446d-a826-3b0a73cc749a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:26:07.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f9352c6f-8d9c-446d-a826-3b0a73cc749a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:26:10.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01de192c-430f-4aa8-a724-6d48c181f3af PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:26:10.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01de192c-430f-4aa8-a724-6d48c181f3af PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@136ffcca 2021-08-11T16:26:10.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01de192c-430f-4aa8-a724-6d48c181f3af PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:26:10.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01de192c-430f-4aa8-a724-6d48c181f3af PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:26:13.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dce1719d-f330-47cb-bf07-60d97b1f1728 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:26:13.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dce1719d-f330-47cb-bf07-60d97b1f1728 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2cdc457b 2021-08-11T16:26:13.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dce1719d-f330-47cb-bf07-60d97b1f1728 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:26:13.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dce1719d-f330-47cb-bf07-60d97b1f1728 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:26:16.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=881ab431-8ec1-4d14-af9b-e47c523c4981 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:26:16.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=881ab431-8ec1-4d14-af9b-e47c523c4981 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4ebaa767 2021-08-11T16:26:16.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=881ab431-8ec1-4d14-af9b-e47c523c4981 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:26:16.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=881ab431-8ec1-4d14-af9b-e47c523c4981 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:26:18.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:26:18.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:26:19.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4415a04c-182e-4a35-b2a7-19b372ed2f3a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:26:19.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4415a04c-182e-4a35-b2a7-19b372ed2f3a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3cfa7836 2021-08-11T16:26:19.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4415a04c-182e-4a35-b2a7-19b372ed2f3a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:26:19.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4415a04c-182e-4a35-b2a7-19b372ed2f3a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:26:22.884Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40be1e7b-1c11-4b09-b936-35d294e6efd1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:26:22.884Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40be1e7b-1c11-4b09-b936-35d294e6efd1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@509f4acf 2021-08-11T16:26:22.885Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40be1e7b-1c11-4b09-b936-35d294e6efd1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:26:22.885Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40be1e7b-1c11-4b09-b936-35d294e6efd1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:26:25.693Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5ee0acb-75b6-474c-9316-4f11a3f2843e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:26:25.693Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5ee0acb-75b6-474c-9316-4f11a3f2843e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1e262a3a 2021-08-11T16:26:25.693Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5ee0acb-75b6-474c-9316-4f11a3f2843e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:26:25.693Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5ee0acb-75b6-474c-9316-4f11a3f2843e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:26:28.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87141bba-6fca-4654-b13f-cebef8b63177 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:26:28.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87141bba-6fca-4654-b13f-cebef8b63177 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5bc3706 2021-08-11T16:26:28.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87141bba-6fca-4654-b13f-cebef8b63177 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:26:28.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=87141bba-6fca-4654-b13f-cebef8b63177 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:26:31.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c000c89-0fb3-40f8-b168-29f57bb39163 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:26:31.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c000c89-0fb3-40f8-b168-29f57bb39163 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6ae8c44f 2021-08-11T16:26:31.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c000c89-0fb3-40f8-b168-29f57bb39163 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:26:31.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c000c89-0fb3-40f8-b168-29f57bb39163 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:26:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:26:33.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:26:34.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d512e913-29e3-4781-980b-51e9df711ebc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:26:34.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d512e913-29e3-4781-980b-51e9df711ebc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@408b32ef 2021-08-11T16:26:34.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d512e913-29e3-4781-980b-51e9df711ebc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:26:34.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d512e913-29e3-4781-980b-51e9df711ebc PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:26:37.632Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18c83a31-a1fe-4fb8-8952-8ab9a5b705f1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:26:37.632Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18c83a31-a1fe-4fb8-8952-8ab9a5b705f1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6fd3a73e 2021-08-11T16:26:37.632Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18c83a31-a1fe-4fb8-8952-8ab9a5b705f1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:26:37.632Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18c83a31-a1fe-4fb8-8952-8ab9a5b705f1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:26:40.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21580c1b-dcf1-40e6-bc78-9096d8dd2d58 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:26:40.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21580c1b-dcf1-40e6-bc78-9096d8dd2d58 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@b33b9c5 2021-08-11T16:26:40.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21580c1b-dcf1-40e6-bc78-9096d8dd2d58 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:26:40.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21580c1b-dcf1-40e6-bc78-9096d8dd2d58 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:26:43.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=716b1827-b622-4344-8576-42cf2474bed2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:26:43.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=716b1827-b622-4344-8576-42cf2474bed2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@27d5ce50 2021-08-11T16:26:43.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=716b1827-b622-4344-8576-42cf2474bed2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:26:43.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=716b1827-b622-4344-8576-42cf2474bed2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:26:46.628Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71cead5a-069c-4f2c-baa4-6e4d09a5bf3b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:26:46.628Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71cead5a-069c-4f2c-baa4-6e4d09a5bf3b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3e509c0f 2021-08-11T16:26:46.628Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71cead5a-069c-4f2c-baa4-6e4d09a5bf3b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:26:46.628Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71cead5a-069c-4f2c-baa4-6e4d09a5bf3b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:26:48.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:26:48.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:26:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:26:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:26:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:26:50.357Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8cbead3-9c39-401e-9adc-9d11c179570d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:26:50.357Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8cbead3-9c39-401e-9adc-9d11c179570d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4cc836ac 2021-08-11T16:26:50.357Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8cbead3-9c39-401e-9adc-9d11c179570d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:26:50.357Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8cbead3-9c39-401e-9adc-9d11c179570d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:26:54.174Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ca03c9b-317c-4338-b58c-caae1d1baa3e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:26:54.174Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ca03c9b-317c-4338-b58c-caae1d1baa3e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5aa34361 2021-08-11T16:26:54.174Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ca03c9b-317c-4338-b58c-caae1d1baa3e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:26:54.174Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ca03c9b-317c-4338-b58c-caae1d1baa3e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:26:56.962Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68ff05cb-d1b3-4a2f-b635-c2af62513986 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:26:56.962Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68ff05cb-d1b3-4a2f-b635-c2af62513986 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@538da12b 2021-08-11T16:26:56.962Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68ff05cb-d1b3-4a2f-b635-c2af62513986 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:26:56.962Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68ff05cb-d1b3-4a2f-b635-c2af62513986 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:27:00.168Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=606b678e-c8d1-45d0-84ef-bc7667bd373f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:27:00.168Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=606b678e-c8d1-45d0-84ef-bc7667bd373f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6907efb1 2021-08-11T16:27:00.168Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=606b678e-c8d1-45d0-84ef-bc7667bd373f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:27:00.168Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=606b678e-c8d1-45d0-84ef-bc7667bd373f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:27:02.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=34e6755a-67b7-4fd2-9e05-e6f05726405f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:27:02.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=34e6755a-67b7-4fd2-9e05-e6f05726405f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@64f7d96b 2021-08-11T16:27:02.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=34e6755a-67b7-4fd2-9e05-e6f05726405f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:27:02.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=34e6755a-67b7-4fd2-9e05-e6f05726405f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:27:03.305Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:27:03.311Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:27:04.254Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:27:05.565Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0e31b87-d7db-43d8-8e28-f65cf74fb8af PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:27:05.565Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0e31b87-d7db-43d8-8e28-f65cf74fb8af PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@240f26df 2021-08-11T16:27:05.565Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0e31b87-d7db-43d8-8e28-f65cf74fb8af PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:27:05.565Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0e31b87-d7db-43d8-8e28-f65cf74fb8af PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:27:07.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa2d1265-e671-44fc-b04c-b3e05c5537a9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:27:07.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa2d1265-e671-44fc-b04c-b3e05c5537a9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1a355f8c 2021-08-11T16:27:07.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa2d1265-e671-44fc-b04c-b3e05c5537a9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:27:07.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa2d1265-e671-44fc-b04c-b3e05c5537a9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:27:10.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0129eb8-52b7-499c-a628-41b88d71b8ed PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:27:10.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0129eb8-52b7-499c-a628-41b88d71b8ed PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4c99511c 2021-08-11T16:27:10.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0129eb8-52b7-499c-a628-41b88d71b8ed PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:27:10.778Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0129eb8-52b7-499c-a628-41b88d71b8ed PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:27:13.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8c56701-7a8d-4310-ab5c-2ede314b904b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:27:13.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8c56701-7a8d-4310-ab5c-2ede314b904b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@167fe565 2021-08-11T16:27:13.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8c56701-7a8d-4310-ab5c-2ede314b904b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:27:13.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8c56701-7a8d-4310-ab5c-2ede314b904b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:27:17.362Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=80ed26b1-027d-4be6-9787-55292fcc92b8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:27:17.362Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=80ed26b1-027d-4be6-9787-55292fcc92b8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4b1b10ab 2021-08-11T16:27:17.362Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=80ed26b1-027d-4be6-9787-55292fcc92b8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:27:17.362Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=80ed26b1-027d-4be6-9787-55292fcc92b8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:27:18.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:27:18.318Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:27:19.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec515060-93a1-46ad-95ab-13de805057aa PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:27:19.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec515060-93a1-46ad-95ab-13de805057aa PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7be2ff7b 2021-08-11T16:27:19.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec515060-93a1-46ad-95ab-13de805057aa PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:27:19.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec515060-93a1-46ad-95ab-13de805057aa PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:27:22.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=002eb8e9-bde7-4613-af9d-ba5a6aebca09 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:27:22.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=002eb8e9-bde7-4613-af9d-ba5a6aebca09 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7d7540aa 2021-08-11T16:27:22.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=002eb8e9-bde7-4613-af9d-ba5a6aebca09 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:27:22.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=002eb8e9-bde7-4613-af9d-ba5a6aebca09 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:27:25.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed327991-5922-45ca-9806-34698ff5cda2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:27:25.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed327991-5922-45ca-9806-34698ff5cda2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1448d6ac 2021-08-11T16:27:25.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed327991-5922-45ca-9806-34698ff5cda2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:27:25.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed327991-5922-45ca-9806-34698ff5cda2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:27:28.951Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1f6bb70-ec20-4fa8-ba4c-e7ac79fa691c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:27:28.951Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1f6bb70-ec20-4fa8-ba4c-e7ac79fa691c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7c502c6 2021-08-11T16:27:28.951Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1f6bb70-ec20-4fa8-ba4c-e7ac79fa691c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:27:28.951Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1f6bb70-ec20-4fa8-ba4c-e7ac79fa691c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:27:31.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=019bc584-be12-4ffc-b66f-86bbd2ea6436 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:27:31.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=019bc584-be12-4ffc-b66f-86bbd2ea6436 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6b303c62 2021-08-11T16:27:31.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=019bc584-be12-4ffc-b66f-86bbd2ea6436 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:27:31.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=019bc584-be12-4ffc-b66f-86bbd2ea6436 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:27:33.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:27:33.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:27:34.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1341de09-515c-4353-a043-d4b56be43774 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:27:34.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1341de09-515c-4353-a043-d4b56be43774 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3c9741be 2021-08-11T16:27:34.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1341de09-515c-4353-a043-d4b56be43774 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:27:34.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1341de09-515c-4353-a043-d4b56be43774 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:27:37.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25e2ed3e-7626-4fbd-912d-ff3025a2945b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:27:37.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25e2ed3e-7626-4fbd-912d-ff3025a2945b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6e380d36 2021-08-11T16:27:37.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25e2ed3e-7626-4fbd-912d-ff3025a2945b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:27:37.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25e2ed3e-7626-4fbd-912d-ff3025a2945b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:27:40.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1eb676bb-7db2-4834-93da-a6d8aaf350db PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:27:40.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1eb676bb-7db2-4834-93da-a6d8aaf350db PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@41c4e105 2021-08-11T16:27:40.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1eb676bb-7db2-4834-93da-a6d8aaf350db PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:27:40.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1eb676bb-7db2-4834-93da-a6d8aaf350db PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:27:43.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41b81b33-92ff-4f31-9869-937f78b3b2bb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:27:43.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41b81b33-92ff-4f31-9869-937f78b3b2bb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2b7d1673 2021-08-11T16:27:43.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41b81b33-92ff-4f31-9869-937f78b3b2bb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:27:43.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41b81b33-92ff-4f31-9869-937f78b3b2bb PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:27:46.886Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f5a55246-1f8c-45bb-8688-cac791a226c7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:27:46.886Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f5a55246-1f8c-45bb-8688-cac791a226c7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2d613813 2021-08-11T16:27:46.886Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f5a55246-1f8c-45bb-8688-cac791a226c7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:27:46.886Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f5a55246-1f8c-45bb-8688-cac791a226c7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:27:48.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:27:48.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:27:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:27:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:27:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:27:50.250Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41a3ae41-baab-4409-a7ef-2e4a963180d6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:27:50.250Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41a3ae41-baab-4409-a7ef-2e4a963180d6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@39ec134 2021-08-11T16:27:50.251Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41a3ae41-baab-4409-a7ef-2e4a963180d6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:27:50.251Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41a3ae41-baab-4409-a7ef-2e4a963180d6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:27:53.250Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f5be2df-de09-4620-8d82-faa99a8441aa PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:27:53.250Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f5be2df-de09-4620-8d82-faa99a8441aa PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6534c48b 2021-08-11T16:27:53.250Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f5be2df-de09-4620-8d82-faa99a8441aa PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:27:53.250Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f5be2df-de09-4620-8d82-faa99a8441aa PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:27:56.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6dab9903-dbd4-4ffa-8ee3-5260321afb53 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:27:56.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6dab9903-dbd4-4ffa-8ee3-5260321afb53 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@263988d0 2021-08-11T16:27:56.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6dab9903-dbd4-4ffa-8ee3-5260321afb53 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:27:56.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6dab9903-dbd4-4ffa-8ee3-5260321afb53 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:27:59.461Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5d78023-4f83-4cb3-bdf3-e82a8d98985b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:27:59.461Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5d78023-4f83-4cb3-bdf3-e82a8d98985b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@23c9da4 2021-08-11T16:27:59.461Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5d78023-4f83-4cb3-bdf3-e82a8d98985b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:27:59.461Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5d78023-4f83-4cb3-bdf3-e82a8d98985b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:28:01.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=890d2459-57d9-4186-a2d4-62a9f9186498 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:28:01.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=890d2459-57d9-4186-a2d4-62a9f9186498 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@775e6bb6 2021-08-11T16:28:01.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=890d2459-57d9-4186-a2d4-62a9f9186498 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:28:01.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=890d2459-57d9-4186-a2d4-62a9f9186498 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:28:03.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:28:03.319Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:28:04.262Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:28:04.601Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21abf96c-df5c-4e4b-b670-b9a406bc8a16 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:28:04.601Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21abf96c-df5c-4e4b-b670-b9a406bc8a16 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@348df94a 2021-08-11T16:28:04.601Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21abf96c-df5c-4e4b-b670-b9a406bc8a16 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:28:04.601Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21abf96c-df5c-4e4b-b670-b9a406bc8a16 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:28:07.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a6cba8f-4419-4361-9989-c9fc4eaf3d81 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:28:07.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a6cba8f-4419-4361-9989-c9fc4eaf3d81 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5e988803 2021-08-11T16:28:07.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a6cba8f-4419-4361-9989-c9fc4eaf3d81 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:28:07.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a6cba8f-4419-4361-9989-c9fc4eaf3d81 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:28:10.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75d7a543-ccac-4d64-9bfd-34d4b45da334 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:28:10.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75d7a543-ccac-4d64-9bfd-34d4b45da334 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@38afefc9 2021-08-11T16:28:10.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75d7a543-ccac-4d64-9bfd-34d4b45da334 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:28:10.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75d7a543-ccac-4d64-9bfd-34d4b45da334 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:28:13.697Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d23d0ac3-b9da-4321-9e2f-22da890a1042 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:28:13.697Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d23d0ac3-b9da-4321-9e2f-22da890a1042 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3ca820b1 2021-08-11T16:28:13.697Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d23d0ac3-b9da-4321-9e2f-22da890a1042 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:28:13.697Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d23d0ac3-b9da-4321-9e2f-22da890a1042 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:28:16.654Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d58ece72-dc46-45d4-99a5-52894298154f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:28:16.654Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d58ece72-dc46-45d4-99a5-52894298154f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@67b4d9c7 2021-08-11T16:28:16.654Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d58ece72-dc46-45d4-99a5-52894298154f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:28:16.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d58ece72-dc46-45d4-99a5-52894298154f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:28:18.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:28:18.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:28:19.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=81997eb1-6f21-476b-83e2-25576086039d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:28:19.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=81997eb1-6f21-476b-83e2-25576086039d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7df68da2 2021-08-11T16:28:19.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=81997eb1-6f21-476b-83e2-25576086039d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:28:19.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=81997eb1-6f21-476b-83e2-25576086039d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:28:22.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=799239f8-998d-4a29-93ee-cff5b6dd52f0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:28:22.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=799239f8-998d-4a29-93ee-cff5b6dd52f0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3ae0f472 2021-08-11T16:28:22.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=799239f8-998d-4a29-93ee-cff5b6dd52f0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:28:22.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=799239f8-998d-4a29-93ee-cff5b6dd52f0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:28:25.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eb720390-a75f-4cf8-b580-cd99af3a746e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:28:25.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eb720390-a75f-4cf8-b580-cd99af3a746e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@47b0e637 2021-08-11T16:28:25.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eb720390-a75f-4cf8-b580-cd99af3a746e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:28:25.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eb720390-a75f-4cf8-b580-cd99af3a746e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:28:28.951Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fc8a70a3-6462-48b0-af2a-45dc18ec226e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:28:28.951Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fc8a70a3-6462-48b0-af2a-45dc18ec226e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@68b75e7 2021-08-11T16:28:28.951Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fc8a70a3-6462-48b0-af2a-45dc18ec226e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:28:28.951Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fc8a70a3-6462-48b0-af2a-45dc18ec226e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:28:31.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cca0a7a1-037b-4c16-9718-b4daf62a2abe PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:28:31.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cca0a7a1-037b-4c16-9718-b4daf62a2abe PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4fd97309 2021-08-11T16:28:31.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cca0a7a1-037b-4c16-9718-b4daf62a2abe PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:28:31.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cca0a7a1-037b-4c16-9718-b4daf62a2abe PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:28:33.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:28:33.316Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:28:34.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=474a96a2-ea5d-49d8-84d2-0e55fb8c1866 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:28:34.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=474a96a2-ea5d-49d8-84d2-0e55fb8c1866 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@72274de4 2021-08-11T16:28:34.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=474a96a2-ea5d-49d8-84d2-0e55fb8c1866 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:28:34.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=474a96a2-ea5d-49d8-84d2-0e55fb8c1866 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:28:37.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a3e70bb-8e4f-466c-ad8e-e8f4a6e7b7c2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:28:37.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a3e70bb-8e4f-466c-ad8e-e8f4a6e7b7c2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@32669cc9 2021-08-11T16:28:37.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a3e70bb-8e4f-466c-ad8e-e8f4a6e7b7c2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:28:37.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a3e70bb-8e4f-466c-ad8e-e8f4a6e7b7c2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:28:40.605Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01d26e6a-8033-444e-b8f5-7f9035eb1790 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:28:40.605Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01d26e6a-8033-444e-b8f5-7f9035eb1790 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6f1fe32f 2021-08-11T16:28:40.606Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01d26e6a-8033-444e-b8f5-7f9035eb1790 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:28:40.606Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01d26e6a-8033-444e-b8f5-7f9035eb1790 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:28:43.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95066692-a5fe-4800-9f1d-99ec8fecd5e0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:28:43.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95066692-a5fe-4800-9f1d-99ec8fecd5e0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6b5a84fe 2021-08-11T16:28:43.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95066692-a5fe-4800-9f1d-99ec8fecd5e0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:28:43.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95066692-a5fe-4800-9f1d-99ec8fecd5e0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:28:46.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f9c7977c-1222-4d47-9dc0-fc2e1f6831f6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:28:46.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f9c7977c-1222-4d47-9dc0-fc2e1f6831f6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@235abc1c 2021-08-11T16:28:46.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f9c7977c-1222-4d47-9dc0-fc2e1f6831f6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:28:46.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f9c7977c-1222-4d47-9dc0-fc2e1f6831f6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:28:48.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:28:48.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:28:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:28:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:28:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:28:50.448Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1c89bbb0-e3c9-4910-aeb7-7342ee20adaa PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:28:50.449Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1c89bbb0-e3c9-4910-aeb7-7342ee20adaa PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@255cfc6a 2021-08-11T16:28:50.449Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1c89bbb0-e3c9-4910-aeb7-7342ee20adaa PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:28:50.449Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1c89bbb0-e3c9-4910-aeb7-7342ee20adaa PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:28:53.477Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a5c3a613-82c0-4881-804e-dc37762206ec PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:28:53.478Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a5c3a613-82c0-4881-804e-dc37762206ec PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@354a6575 2021-08-11T16:28:53.478Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a5c3a613-82c0-4881-804e-dc37762206ec PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:28:53.478Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a5c3a613-82c0-4881-804e-dc37762206ec PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:28:56.965Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29f98383-bfa0-4211-95ef-af243b0f8e51 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:28:56.965Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29f98383-bfa0-4211-95ef-af243b0f8e51 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@a086a56 2021-08-11T16:28:56.965Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29f98383-bfa0-4211-95ef-af243b0f8e51 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:28:56.965Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29f98383-bfa0-4211-95ef-af243b0f8e51 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:28:59.253Z [cluster1-nio-worker-1] WARN c.d.driver.core.HostConnectionPool Error creating connection to cassandra.onap/10.242.184.131:9042 com.datastax.driver.core.exceptions.TransportException: [cassandra.onap/10.242.184.131:9042] Connection has been closed at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1427) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1408) at com.datastax.driver.core.Connection.defunct(Connection.java:585) at com.datastax.driver.core.Connection$Dispatcher.exceptionCaught(Connection.java:1330) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273) at io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.handleReadException(AbstractEpollStreamChannel.java:728) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:818) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-08-11T16:28:59.253Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.184.131:9042 com.datastax.driver.core.exceptions.ConnectionException: [cassandra.onap/10.242.184.131:9042] Pool was closed during initialization at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:173) at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:158) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1080) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.google.common.util.concurrent.CollectionFuture.handleAllCompleted(CollectionFuture.java:69) at com.google.common.util.concurrent.AggregateFuture.processCompleted(AggregateFuture.java:282) at com.google.common.util.concurrent.AggregateFuture.decrementCountAndMaybeComplete(AggregateFuture.java:264) at com.google.common.util.concurrent.AggregateFuture.access$200(AggregateFuture.java:42) at com.google.common.util.concurrent.AggregateFuture$1.run(AggregateFuture.java:147) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:100) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.datastax.driver.core.Connection$Future.onException(Connection.java:1518) at com.datastax.driver.core.Connection$Dispatcher.errorOutAllHandler(Connection.java:1340) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1427) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1408) at com.datastax.driver.core.Connection.defunct(Connection.java:585) at com.datastax.driver.core.Connection$Dispatcher.exceptionCaught(Connection.java:1330) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273) at io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.handleReadException(AbstractEpollStreamChannel.java:728) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:818) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-08-11T16:29:00.662Z [cluster1-nio-worker-1] WARN c.d.driver.core.HostConnectionPool Error creating connection to cassandra.onap/10.242.110.137:9042 com.datastax.driver.core.exceptions.TransportException: [cassandra.onap/10.242.110.137:9042] Connection has been closed at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1427) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1408) at com.datastax.driver.core.Connection.defunct(Connection.java:585) at com.datastax.driver.core.Connection$Dispatcher.exceptionCaught(Connection.java:1330) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273) at io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.handleReadException(AbstractEpollStreamChannel.java:728) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:818) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-08-11T16:29:00.663Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.110.137:9042 com.datastax.driver.core.exceptions.ConnectionException: [cassandra.onap/10.242.110.137:9042] Pool was closed during initialization at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:173) at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:158) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1080) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.google.common.util.concurrent.CollectionFuture.handleAllCompleted(CollectionFuture.java:69) at com.google.common.util.concurrent.AggregateFuture.processCompleted(AggregateFuture.java:282) at com.google.common.util.concurrent.AggregateFuture.decrementCountAndMaybeComplete(AggregateFuture.java:264) at com.google.common.util.concurrent.AggregateFuture.access$200(AggregateFuture.java:42) at com.google.common.util.concurrent.AggregateFuture$1.run(AggregateFuture.java:147) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:100) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.datastax.driver.core.Connection$Future.onException(Connection.java:1518) at com.datastax.driver.core.Connection$Dispatcher.errorOutAllHandler(Connection.java:1340) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1427) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1408) at com.datastax.driver.core.Connection.defunct(Connection.java:585) at com.datastax.driver.core.Connection$Dispatcher.exceptionCaught(Connection.java:1330) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273) at io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.handleReadException(AbstractEpollStreamChannel.java:728) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:818) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-08-11T16:29:00.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf9ab446-1d49-4503-8a29-d25d051bd59b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:29:00.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf9ab446-1d49-4503-8a29-d25d051bd59b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7cc48d57 2021-08-11T16:29:00.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf9ab446-1d49-4503-8a29-d25d051bd59b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:29:00.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf9ab446-1d49-4503-8a29-d25d051bd59b PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:29:00.683Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=cf9ab446-1d49-4503-8a29-d25d051bd59b ErrorCategory=INFO ServiceName=BE Component CASSANDRA state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component CASSANDRA state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:29:00.683Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=cf9ab446-1d49-4503-8a29-d25d051bd59b ErrorCategory=ERROR ServiceName=BE Component CASSANDRA state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-08-11T16:29:00.950Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:00.978Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:01.034Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:01.086Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:01.138Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:01.190Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:01.293Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:01.495Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:01.595Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3bdde239-f866-4291-9b36-e7f2d250ff89 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:29:01.595Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3bdde239-f866-4291-9b36-e7f2d250ff89 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3eb99109 2021-08-11T16:29:01.595Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3bdde239-f866-4291-9b36-e7f2d250ff89 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:29:01.595Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3bdde239-f866-4291-9b36-e7f2d250ff89 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:29:01.898Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:02.246Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:02.299Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:02.300Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:02.401Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:02.504Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:02.606Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:02.708Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:02.811Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:02.913Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:03.016Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:03.103Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:03.118Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:03.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:29:03.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-08-11T16:29:03.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:29:03.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:29:03.321Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:03.532Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:03.734Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:03.937Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:04.140Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:04.279Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:29:04.542Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:04.576Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba941d20-e026-4dd2-85ea-dbe6cf309e3e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:29:04.576Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba941d20-e026-4dd2-85ea-dbe6cf309e3e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@58be2a66 2021-08-11T16:29:04.576Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba941d20-e026-4dd2-85ea-dbe6cf309e3e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:29:04.576Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba941d20-e026-4dd2-85ea-dbe6cf309e3e PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:29:04.590Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=ba941d20-e026-4dd2-85ea-dbe6cf309e3e ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:29:04.590Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=ba941d20-e026-4dd2-85ea-dbe6cf309e3e ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-08-11T16:29:04.706Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T16:28:55.378972Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:05.345Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:06.147Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:06.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:29:07.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9bffca9b-841a-4902-91fa-3e98164343fd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:29:07.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9bffca9b-841a-4902-91fa-3e98164343fd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@11adb1fd 2021-08-11T16:29:07.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9bffca9b-841a-4902-91fa-3e98164343fd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:29:07.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9bffca9b-841a-4902-91fa-3e98164343fd PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:29:07.750Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:09.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:29:09.353Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:09.709Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:09.761Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:09.863Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:09.966Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:10.068Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:10.171Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:10.273Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:10.376Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:10.478Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:10.581Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:10.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa10ff4e-5cae-4c72-adfd-47547a0c9e23 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:29:10.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa10ff4e-5cae-4c72-adfd-47547a0c9e23 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@495b38aa 2021-08-11T16:29:10.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa10ff4e-5cae-4c72-adfd-47547a0c9e23 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:29:10.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa10ff4e-5cae-4c72-adfd-47547a0c9e23 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:29:10.783Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:10.956Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:10.984Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:10.998Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:11.036Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:11.088Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:11.190Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:11.393Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:11.400Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:11.796Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:11.802Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:12.199Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:12.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:29:12.605Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T16:28:55.378972Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:13.002Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:13.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29847b94-f342-4853-b6b7-d3a16df80734 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:29:13.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29847b94-f342-4853-b6b7-d3a16df80734 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3b18c951 2021-08-11T16:29:13.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29847b94-f342-4853-b6b7-d3a16df80734 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:29:13.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29847b94-f342-4853-b6b7-d3a16df80734 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:29:14.605Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:15.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:29:16.606Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c640d978-f34f-4a2d-a136-30d031a1fea3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:29:16.606Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c640d978-f34f-4a2d-a136-30d031a1fea3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@12dc294e 2021-08-11T16:29:16.606Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c640d978-f34f-4a2d-a136-30d031a1fea3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:29:16.606Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c640d978-f34f-4a2d-a136-30d031a1fea3 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:29:17.609Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:17.637Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:17.664Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:17.692Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:17.745Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:17.798Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:17.809Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:17.837Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:17.850Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:17.865Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:17.892Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:17.919Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:17.947Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:17.952Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:18.000Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:18.052Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:18.055Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:18.155Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:18.246Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:29:18.258Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:308) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:422) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:18.259Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:18.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:29:18.318Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:29:18.463Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:18.661Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:18.866Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:19.064Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:19.268Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:19.467Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:19.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29276ed1-29f2-4ef0-ac5b-b5d622952bce PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:29:19.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29276ed1-29f2-4ef0-ac5b-b5d622952bce PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6e0240b1 2021-08-11T16:29:19.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29276ed1-29f2-4ef0-ac5b-b5d622952bce PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:29:19.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29276ed1-29f2-4ef0-ac5b-b5d622952bce PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:29:20.072Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:20.270Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:20.875Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:21.073Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T16:28:55.378972Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:21.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:29:21.678Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:22.480Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:22.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f4c1ded-28e9-4b3e-86a0-a26a39f50166 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:29:22.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f4c1ded-28e9-4b3e-86a0-a26a39f50166 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7e8cbbb3 2021-08-11T16:29:22.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f4c1ded-28e9-4b3e-86a0-a26a39f50166 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:29:22.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f4c1ded-28e9-4b3e-86a0-a26a39f50166 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:29:23.283Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:24.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:29:24.886Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:25.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=22b3a2a2-4920-4324-8e64-011748c3eedd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:29:25.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=22b3a2a2-4920-4324-8e64-011748c3eedd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3a3a772 2021-08-11T16:29:25.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=22b3a2a2-4920-4324-8e64-011748c3eedd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:29:25.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=22b3a2a2-4920-4324-8e64-011748c3eedd PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:29:26.077Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:26.148Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:26.189Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:26.248Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:26.348Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:26.450Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:26.550Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:26.647Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:26.652Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:26.700Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:26.848Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:26.950Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:27.055Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:27.153Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:27.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:29:27.355Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:27.558Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:27.858Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:27.961Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:28.364Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:28.848Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:28.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d879b557-de2d-427e-8e5f-a5c38a942f48 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:29:28.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d879b557-de2d-427e-8e5f-a5c38a942f48 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@70466811 2021-08-11T16:29:28.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d879b557-de2d-427e-8e5f-a5c38a942f48 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:29:28.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d879b557-de2d-427e-8e5f-a5c38a942f48 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:29:29.460Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T16:28:55.378972Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:29.651Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:30.245Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:29:30.453Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:31.577Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3cf4a0d-ee27-4bce-90b6-fc412898b6b3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:29:31.577Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3cf4a0d-ee27-4bce-90b6-fc412898b6b3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@c3da960 2021-08-11T16:29:31.577Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3cf4a0d-ee27-4bce-90b6-fc412898b6b3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:29:31.577Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3cf4a0d-ee27-4bce-90b6-fc412898b6b3 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:29:32.057Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:33.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:29:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:29:33.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:29:33.661Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:33.714Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:33.816Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:34.020Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:34.222Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:34.463Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:34.516Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:34.568Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:34.586Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d7862e82-5cfc-48c0-b3c8-2ded407eb3e5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:29:34.586Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d7862e82-5cfc-48c0-b3c8-2ded407eb3e5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1777da77 2021-08-11T16:29:34.586Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d7862e82-5cfc-48c0-b3c8-2ded407eb3e5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:29:34.586Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d7862e82-5cfc-48c0-b3c8-2ded407eb3e5 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:29:34.620Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:34.625Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:34.723Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:34.825Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:35.028Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:35.028Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:35.230Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:35.431Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:35.633Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:35.833Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:36.036Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:36.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:29:36.637Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:36.839Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:37.648Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:37.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6dfcba55-b948-4e4f-98da-54165b7fdfed PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:29:37.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6dfcba55-b948-4e4f-98da-54165b7fdfed PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@9f177dc 2021-08-11T16:29:37.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6dfcba55-b948-4e4f-98da-54165b7fdfed PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:29:37.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6dfcba55-b948-4e4f-98da-54165b7fdfed PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:29:38.248Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:38.451Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T16:28:55.378972Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:39.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:29:40.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df3b2b01-2d0b-441f-a9f7-aac0456f6550 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:29:40.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df3b2b01-2d0b-441f-a9f7-aac0456f6550 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4ab49013 2021-08-11T16:29:40.867Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df3b2b01-2d0b-441f-a9f7-aac0456f6550 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:29:40.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df3b2b01-2d0b-441f-a9f7-aac0456f6550 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:29:41.454Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:41.482Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:41.535Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:41.587Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:41.690Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:41.892Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:42.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:29:42.295Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:43.099Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:43.455Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:43.483Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:43.512Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:43.548Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:43.600Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:43.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9241812d-fdde-4f7f-8099-b31fd23c0f24 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:29:43.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9241812d-fdde-4f7f-8099-b31fd23c0f24 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2b23468c 2021-08-11T16:29:43.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9241812d-fdde-4f7f-8099-b31fd23c0f24 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:29:43.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9241812d-fdde-4f7f-8099-b31fd23c0f24 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:29:43.653Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:43.705Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:43.809Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:43.901Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:44.012Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:29:45.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:29:46.574Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=416d9dfe-94e3-45d7-815f-fab4a4721a33 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:29:46.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=416d9dfe-94e3-45d7-815f-fab4a4721a33 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@22e95fd 2021-08-11T16:29:46.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=416d9dfe-94e3-45d7-815f-fab4a4721a33 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:29:46.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=416d9dfe-94e3-45d7-815f-fab4a4721a33 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:29:47.260Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-08-11T16:29:48.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:29:48.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:29:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:29:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:29:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:29:49.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6dcbf750-dafe-4516-b997-67f5f6ba229f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:29:49.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6dcbf750-dafe-4516-b997-67f5f6ba229f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@e34df3 2021-08-11T16:29:49.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6dcbf750-dafe-4516-b997-67f5f6ba229f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:29:49.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6dcbf750-dafe-4516-b997-67f5f6ba229f PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:29:49.976Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=6dcbf750-dafe-4516-b997-67f5f6ba229f ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:29:49.976Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=6dcbf750-dafe-4516-b997-67f5f6ba229f ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-08-11T16:29:52.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10ca4196-4f65-46dd-ac21-00b169d9ca42 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:29:52.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10ca4196-4f65-46dd-ac21-00b169d9ca42 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6e3961f4 2021-08-11T16:29:52.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10ca4196-4f65-46dd-ac21-00b169d9ca42 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:29:52.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10ca4196-4f65-46dd-ac21-00b169d9ca42 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:29:55.605Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=46ccbbc0-3238-4a31-8de0-bbb78089b6a1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:29:55.605Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=46ccbbc0-3238-4a31-8de0-bbb78089b6a1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5f9f5d99 2021-08-11T16:29:55.605Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=46ccbbc0-3238-4a31-8de0-bbb78089b6a1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:29:55.605Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=46ccbbc0-3238-4a31-8de0-bbb78089b6a1 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:30:01.552Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a5cba3b-9fec-434a-b780-88613277acc5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:30:01.552Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a5cba3b-9fec-434a-b780-88613277acc5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2968baea 2021-08-11T16:30:01.552Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a5cba3b-9fec-434a-b780-88613277acc5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:30:01.552Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a5cba3b-9fec-434a-b780-88613277acc5 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:30:03.320Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:30:03.327Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:30:03.966Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=5a5cba3b-9fec-434a-b780-88613277acc5 ErrorCategory=INFO ServiceName=BE Component CASSANDRA state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component CASSANDRA state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:30:03.967Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=5a5cba3b-9fec-434a-b780-88613277acc5 ErrorCategory=INFO ServiceName=BE Component CASSANDRA state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:30:04.276Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:30:04.759Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df99c7ec-e966-4931-9097-1f0f974ac66f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:30:04.759Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df99c7ec-e966-4931-9097-1f0f974ac66f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@337ef6e1 2021-08-11T16:30:04.759Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df99c7ec-e966-4931-9097-1f0f974ac66f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:30:04.759Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df99c7ec-e966-4931-9097-1f0f974ac66f PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:30:06.352Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0158e621-fa29-49f4-83d3-632383069aa8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:30:06.352Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0158e621-fa29-49f4-83d3-632383069aa8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@16ed07bb 2021-08-11T16:30:06.352Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0158e621-fa29-49f4-83d3-632383069aa8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:30:06.352Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0158e621-fa29-49f4-83d3-632383069aa8 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:30:07.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f724f40-b899-4cc2-b6e4-da68e28e78ab PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:30:07.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f724f40-b899-4cc2-b6e4-da68e28e78ab PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@33bbf36a 2021-08-11T16:30:07.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f724f40-b899-4cc2-b6e4-da68e28e78ab PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:30:07.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f724f40-b899-4cc2-b6e4-da68e28e78ab PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:30:11.146Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a604f4f-95f2-43e0-9bec-f8ee1a5cc535 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:30:11.146Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a604f4f-95f2-43e0-9bec-f8ee1a5cc535 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5fe8b01d 2021-08-11T16:30:11.146Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a604f4f-95f2-43e0-9bec-f8ee1a5cc535 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:30:11.146Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a604f4f-95f2-43e0-9bec-f8ee1a5cc535 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:30:13.851Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=998a1acf-67e9-4e21-81bd-2dd98943253a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:30:13.851Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=998a1acf-67e9-4e21-81bd-2dd98943253a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1be58cff 2021-08-11T16:30:13.851Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=998a1acf-67e9-4e21-81bd-2dd98943253a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:30:13.851Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=998a1acf-67e9-4e21-81bd-2dd98943253a PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:30:16.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0358b378-c9b0-40d7-8c9a-558bf663e2f5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:30:16.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0358b378-c9b0-40d7-8c9a-558bf663e2f5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@65cb90e7 2021-08-11T16:30:16.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0358b378-c9b0-40d7-8c9a-558bf663e2f5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:30:16.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0358b378-c9b0-40d7-8c9a-558bf663e2f5 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:30:18.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:30:18.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:30:19.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=407bb46a-efa0-4614-87e8-ccbbf05828d7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:30:19.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=407bb46a-efa0-4614-87e8-ccbbf05828d7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@9871f97 2021-08-11T16:30:19.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=407bb46a-efa0-4614-87e8-ccbbf05828d7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:30:19.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=407bb46a-efa0-4614-87e8-ccbbf05828d7 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:30:22.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66f4c38d-d2a9-4ce2-b5da-e462403738cd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:30:22.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66f4c38d-d2a9-4ce2-b5da-e462403738cd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@13f52ee2 2021-08-11T16:30:22.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66f4c38d-d2a9-4ce2-b5da-e462403738cd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:30:22.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66f4c38d-d2a9-4ce2-b5da-e462403738cd PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:30:25.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a98f319d-86fd-4440-abb3-18b08ff2e72b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:30:25.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a98f319d-86fd-4440-abb3-18b08ff2e72b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@24c0da17 2021-08-11T16:30:25.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a98f319d-86fd-4440-abb3-18b08ff2e72b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:30:25.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a98f319d-86fd-4440-abb3-18b08ff2e72b PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:30:28.950Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=45083039-971f-48b0-9708-8902ff994dfe PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:30:28.950Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=45083039-971f-48b0-9708-8902ff994dfe PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1db52ac0 2021-08-11T16:30:28.950Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=45083039-971f-48b0-9708-8902ff994dfe PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:30:28.950Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=45083039-971f-48b0-9708-8902ff994dfe PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:30:31.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85b221bf-e39f-49c9-bec6-dc7e13837a87 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:30:31.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85b221bf-e39f-49c9-bec6-dc7e13837a87 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2cffe546 2021-08-11T16:30:31.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85b221bf-e39f-49c9-bec6-dc7e13837a87 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:30:31.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85b221bf-e39f-49c9-bec6-dc7e13837a87 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:30:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:30:33.316Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:30:34.687Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4a68c62-970c-4241-b79a-1d95f9d5a52a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:30:34.688Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4a68c62-970c-4241-b79a-1d95f9d5a52a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@16c74c05 2021-08-11T16:30:34.688Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4a68c62-970c-4241-b79a-1d95f9d5a52a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:30:34.688Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4a68c62-970c-4241-b79a-1d95f9d5a52a PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:30:37.617Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3bb420af-e362-4d50-82b7-f83b3b338e3d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:30:37.617Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3bb420af-e362-4d50-82b7-f83b3b338e3d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@34753ca8 2021-08-11T16:30:37.617Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3bb420af-e362-4d50-82b7-f83b3b338e3d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:30:37.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3bb420af-e362-4d50-82b7-f83b3b338e3d PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:30:40.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=735bf5ee-d0a6-4811-bad2-0761edf234d9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:30:40.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=735bf5ee-d0a6-4811-bad2-0761edf234d9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@a2e3cc 2021-08-11T16:30:40.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=735bf5ee-d0a6-4811-bad2-0761edf234d9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:30:40.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=735bf5ee-d0a6-4811-bad2-0761edf234d9 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:30:43.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95fd3513-2111-45c3-9102-6d84d51e8d24 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:30:43.652Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95fd3513-2111-45c3-9102-6d84d51e8d24 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5324f34b 2021-08-11T16:30:43.652Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95fd3513-2111-45c3-9102-6d84d51e8d24 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:30:43.652Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95fd3513-2111-45c3-9102-6d84d51e8d24 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:30:46.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea0ccd1d-38af-4f33-a1d7-a559e1ccb340 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:30:46.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea0ccd1d-38af-4f33-a1d7-a559e1ccb340 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7faf2a4e 2021-08-11T16:30:46.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea0ccd1d-38af-4f33-a1d7-a559e1ccb340 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:30:46.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea0ccd1d-38af-4f33-a1d7-a559e1ccb340 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:30:48.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:30:48.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:30:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:30:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:30:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:30:49.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=63087173-c0b8-4cdf-a8fb-e96a7edeec7b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:30:49.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=63087173-c0b8-4cdf-a8fb-e96a7edeec7b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6222d96f 2021-08-11T16:30:49.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=63087173-c0b8-4cdf-a8fb-e96a7edeec7b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:30:49.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=63087173-c0b8-4cdf-a8fb-e96a7edeec7b PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:30:52.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d7190b75-2475-4cce-a97e-e68914a18d8b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:30:52.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d7190b75-2475-4cce-a97e-e68914a18d8b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7509fa64 2021-08-11T16:30:52.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d7190b75-2475-4cce-a97e-e68914a18d8b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:30:52.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d7190b75-2475-4cce-a97e-e68914a18d8b PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:30:55.876Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0505cf9e-f64e-4bc1-8470-9dbb465ecf05 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:30:55.876Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0505cf9e-f64e-4bc1-8470-9dbb465ecf05 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@143e125 2021-08-11T16:30:55.876Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0505cf9e-f64e-4bc1-8470-9dbb465ecf05 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:30:55.876Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0505cf9e-f64e-4bc1-8470-9dbb465ecf05 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:30:58.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=60f8db5f-20ea-458a-a673-c4011155e2aa PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:30:58.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=60f8db5f-20ea-458a-a673-c4011155e2aa PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7385619 2021-08-11T16:30:58.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=60f8db5f-20ea-458a-a673-c4011155e2aa PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:30:58.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=60f8db5f-20ea-458a-a673-c4011155e2aa PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:31:01.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18e338fb-9dc7-45ac-b84d-b654da08a039 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:31:01.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18e338fb-9dc7-45ac-b84d-b654da08a039 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@73251e30 2021-08-11T16:31:01.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18e338fb-9dc7-45ac-b84d-b654da08a039 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:31:01.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18e338fb-9dc7-45ac-b84d-b654da08a039 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:31:03.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:31:03.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:31:04.254Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:31:04.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c485d4b2-fcea-417c-ac03-c892bec6ff6d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:31:04.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c485d4b2-fcea-417c-ac03-c892bec6ff6d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@49c500ea 2021-08-11T16:31:04.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c485d4b2-fcea-417c-ac03-c892bec6ff6d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:31:04.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c485d4b2-fcea-417c-ac03-c892bec6ff6d PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:31:07.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9c96413d-ceed-4aae-b50e-fb9387ee0b7b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:31:07.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9c96413d-ceed-4aae-b50e-fb9387ee0b7b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@57119a3e 2021-08-11T16:31:07.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9c96413d-ceed-4aae-b50e-fb9387ee0b7b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:31:07.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9c96413d-ceed-4aae-b50e-fb9387ee0b7b PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:31:10.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0c0dcdfd-fc14-478d-ac2b-6eee8f28615e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:31:10.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0c0dcdfd-fc14-478d-ac2b-6eee8f28615e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@125c865b 2021-08-11T16:31:10.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0c0dcdfd-fc14-478d-ac2b-6eee8f28615e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:31:10.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0c0dcdfd-fc14-478d-ac2b-6eee8f28615e PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:31:13.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d6d111a1-662b-439c-aea9-321a2a3ce2ca PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:31:13.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d6d111a1-662b-439c-aea9-321a2a3ce2ca PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4953debe 2021-08-11T16:31:13.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d6d111a1-662b-439c-aea9-321a2a3ce2ca PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:31:13.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d6d111a1-662b-439c-aea9-321a2a3ce2ca PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:31:16.654Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7daff9c4-ff1f-4e30-9c25-3e7f8388be8c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:31:16.654Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7daff9c4-ff1f-4e30-9c25-3e7f8388be8c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2870452e 2021-08-11T16:31:16.654Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7daff9c4-ff1f-4e30-9c25-3e7f8388be8c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:31:16.654Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7daff9c4-ff1f-4e30-9c25-3e7f8388be8c PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:31:18.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:31:18.318Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:31:19.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a13fc73-37b5-4a9f-8dfd-6229fe86990b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:31:19.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a13fc73-37b5-4a9f-8dfd-6229fe86990b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@736bfa74 2021-08-11T16:31:19.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a13fc73-37b5-4a9f-8dfd-6229fe86990b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:31:19.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a13fc73-37b5-4a9f-8dfd-6229fe86990b PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:31:22.710Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f20c6316-5ec8-43be-ab67-4dd022f489cd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:31:22.710Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f20c6316-5ec8-43be-ab67-4dd022f489cd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@573c41c 2021-08-11T16:31:22.710Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f20c6316-5ec8-43be-ab67-4dd022f489cd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:31:22.710Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f20c6316-5ec8-43be-ab67-4dd022f489cd PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:31:25.749Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ff2693c-c615-417f-b39c-7e8b2c1290b2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:31:25.749Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ff2693c-c615-417f-b39c-7e8b2c1290b2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@52024c22 2021-08-11T16:31:25.749Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ff2693c-c615-417f-b39c-7e8b2c1290b2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:31:25.749Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ff2693c-c615-417f-b39c-7e8b2c1290b2 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:31:28.879Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=60a877e3-4267-43b0-91e4-433ac2a61bd4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:31:28.879Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=60a877e3-4267-43b0-91e4-433ac2a61bd4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@78713a79 2021-08-11T16:31:28.879Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=60a877e3-4267-43b0-91e4-433ac2a61bd4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:31:28.879Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=60a877e3-4267-43b0-91e4-433ac2a61bd4 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:31:31.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=868a7fc1-3a82-4597-9f19-7283367adda9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:31:31.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=868a7fc1-3a82-4597-9f19-7283367adda9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@54e5c759 2021-08-11T16:31:31.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=868a7fc1-3a82-4597-9f19-7283367adda9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:31:31.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=868a7fc1-3a82-4597-9f19-7283367adda9 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:31:33.310Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:31:33.320Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:31:34.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ded3c9ae-3145-4344-84aa-b7d2177445e1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:31:34.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ded3c9ae-3145-4344-84aa-b7d2177445e1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4f2daafd 2021-08-11T16:31:34.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ded3c9ae-3145-4344-84aa-b7d2177445e1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:31:34.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ded3c9ae-3145-4344-84aa-b7d2177445e1 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:31:37.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5efe32a-36fd-4096-9a7e-82fe57beee33 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:31:37.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5efe32a-36fd-4096-9a7e-82fe57beee33 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@f8e9c66 2021-08-11T16:31:37.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5efe32a-36fd-4096-9a7e-82fe57beee33 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:31:37.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5efe32a-36fd-4096-9a7e-82fe57beee33 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:31:40.760Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d450a140-cc0b-4071-ba7e-e5a1d18c17b3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:31:40.760Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d450a140-cc0b-4071-ba7e-e5a1d18c17b3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7e5ddd1c 2021-08-11T16:31:40.760Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d450a140-cc0b-4071-ba7e-e5a1d18c17b3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:31:40.760Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d450a140-cc0b-4071-ba7e-e5a1d18c17b3 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:31:43.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=319401cd-949a-4dfa-9aaa-c7b810be22f3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:31:43.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=319401cd-949a-4dfa-9aaa-c7b810be22f3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4781281 2021-08-11T16:31:43.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=319401cd-949a-4dfa-9aaa-c7b810be22f3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:31:43.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=319401cd-949a-4dfa-9aaa-c7b810be22f3 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:31:46.691Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ccd78d4-2ffd-4801-83d3-670e05b96dea PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:31:46.691Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ccd78d4-2ffd-4801-83d3-670e05b96dea PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@26202ab1 2021-08-11T16:31:46.691Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ccd78d4-2ffd-4801-83d3-670e05b96dea PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:31:46.691Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ccd78d4-2ffd-4801-83d3-670e05b96dea PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:31:48.309Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:31:48.320Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:31:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:31:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:31:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:31:50.156Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9c105d20-d6c9-47c3-be83-aa22ac0a89bc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:31:50.157Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9c105d20-d6c9-47c3-be83-aa22ac0a89bc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5db6efbf 2021-08-11T16:31:50.157Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9c105d20-d6c9-47c3-be83-aa22ac0a89bc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:31:50.157Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9c105d20-d6c9-47c3-be83-aa22ac0a89bc PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:31:53.262Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c2ea328-62da-4fe2-a880-9fd65768e42d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:31:53.262Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c2ea328-62da-4fe2-a880-9fd65768e42d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4d137c28 2021-08-11T16:31:53.262Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c2ea328-62da-4fe2-a880-9fd65768e42d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:31:53.262Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c2ea328-62da-4fe2-a880-9fd65768e42d PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:31:56.354Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa061281-d6fd-47f9-9e89-01dad5ba73c2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:31:56.354Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa061281-d6fd-47f9-9e89-01dad5ba73c2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@93b9311 2021-08-11T16:31:56.355Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa061281-d6fd-47f9-9e89-01dad5ba73c2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:31:56.355Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa061281-d6fd-47f9-9e89-01dad5ba73c2 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:31:58.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7b458bf-2363-402c-a960-9805b8fc3544 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:31:58.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7b458bf-2363-402c-a960-9805b8fc3544 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2e694525 2021-08-11T16:31:58.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7b458bf-2363-402c-a960-9805b8fc3544 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:31:58.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7b458bf-2363-402c-a960-9805b8fc3544 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:32:01.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=022728d4-388b-44a0-9d90-4d7c046be549 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:32:01.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=022728d4-388b-44a0-9d90-4d7c046be549 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@44c89c40 2021-08-11T16:32:01.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=022728d4-388b-44a0-9d90-4d7c046be549 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:32:01.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=022728d4-388b-44a0-9d90-4d7c046be549 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:32:03.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:32:03.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:32:04.262Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:32:04.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2388bb5d-460f-49dc-a49d-f6f8f8b664f6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:32:04.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2388bb5d-460f-49dc-a49d-f6f8f8b664f6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@126c5acd 2021-08-11T16:32:04.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2388bb5d-460f-49dc-a49d-f6f8f8b664f6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:32:04.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2388bb5d-460f-49dc-a49d-f6f8f8b664f6 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:32:07.601Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d87be3e1-2582-4d5f-a751-4703f0e4459d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:32:07.601Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d87be3e1-2582-4d5f-a751-4703f0e4459d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@471ae08a 2021-08-11T16:32:07.601Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d87be3e1-2582-4d5f-a751-4703f0e4459d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:32:07.602Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d87be3e1-2582-4d5f-a751-4703f0e4459d PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:32:10.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1199b215-994f-46fb-85c3-4428fb3caa4c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:32:10.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1199b215-994f-46fb-85c3-4428fb3caa4c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@233ed521 2021-08-11T16:32:10.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1199b215-994f-46fb-85c3-4428fb3caa4c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:32:10.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1199b215-994f-46fb-85c3-4428fb3caa4c PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:32:13.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3e1bbe39-e58d-4059-9182-2c7a251cabe7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:32:13.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3e1bbe39-e58d-4059-9182-2c7a251cabe7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2314779a 2021-08-11T16:32:13.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3e1bbe39-e58d-4059-9182-2c7a251cabe7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:32:13.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3e1bbe39-e58d-4059-9182-2c7a251cabe7 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:32:16.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03daddf9-95ba-493d-94f6-3107255e7951 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:32:16.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03daddf9-95ba-493d-94f6-3107255e7951 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@53deeb07 2021-08-11T16:32:16.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03daddf9-95ba-493d-94f6-3107255e7951 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:32:16.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03daddf9-95ba-493d-94f6-3107255e7951 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:32:18.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:32:18.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:32:19.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58f284d2-1d00-4a0e-9c22-b4a578148e4c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:32:19.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58f284d2-1d00-4a0e-9c22-b4a578148e4c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@43d05f46 2021-08-11T16:32:19.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58f284d2-1d00-4a0e-9c22-b4a578148e4c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:32:19.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58f284d2-1d00-4a0e-9c22-b4a578148e4c PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:32:22.595Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=281a9355-3f0c-4e38-aa5d-e723477d73f8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:32:22.595Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=281a9355-3f0c-4e38-aa5d-e723477d73f8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@16120dc0 2021-08-11T16:32:22.595Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=281a9355-3f0c-4e38-aa5d-e723477d73f8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:32:22.595Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=281a9355-3f0c-4e38-aa5d-e723477d73f8 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:32:25.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=97ebd4eb-1e8e-493f-9848-0c885dda24a0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:32:25.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=97ebd4eb-1e8e-493f-9848-0c885dda24a0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@16ba103f 2021-08-11T16:32:25.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=97ebd4eb-1e8e-493f-9848-0c885dda24a0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:32:25.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=97ebd4eb-1e8e-493f-9848-0c885dda24a0 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:32:28.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=351772d5-30b4-4bad-bc2e-b2df7a0f233e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:32:28.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=351772d5-30b4-4bad-bc2e-b2df7a0f233e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3218573c 2021-08-11T16:32:28.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=351772d5-30b4-4bad-bc2e-b2df7a0f233e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:32:28.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=351772d5-30b4-4bad-bc2e-b2df7a0f233e PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:32:31.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf1abcd3-bbb2-4067-82ab-8fb34d3f4b7c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:32:31.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf1abcd3-bbb2-4067-82ab-8fb34d3f4b7c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@73730456 2021-08-11T16:32:31.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf1abcd3-bbb2-4067-82ab-8fb34d3f4b7c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:32:31.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf1abcd3-bbb2-4067-82ab-8fb34d3f4b7c PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:32:33.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:32:33.317Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:32:34.595Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0e5ebc5-a686-48a8-8ee6-9de74ef81646 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:32:34.595Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0e5ebc5-a686-48a8-8ee6-9de74ef81646 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@75096bf5 2021-08-11T16:32:34.596Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0e5ebc5-a686-48a8-8ee6-9de74ef81646 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:32:34.596Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0e5ebc5-a686-48a8-8ee6-9de74ef81646 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:32:37.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b7e539e-8b8d-49fc-b8ba-32e021180868 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:32:37.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b7e539e-8b8d-49fc-b8ba-32e021180868 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3374a683 2021-08-11T16:32:37.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b7e539e-8b8d-49fc-b8ba-32e021180868 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:32:37.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b7e539e-8b8d-49fc-b8ba-32e021180868 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:32:40.608Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e2180b8-b716-4f46-8eae-1b7cd0f2294d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:32:40.608Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e2180b8-b716-4f46-8eae-1b7cd0f2294d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@43a5a666 2021-08-11T16:32:40.608Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e2180b8-b716-4f46-8eae-1b7cd0f2294d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:32:40.609Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e2180b8-b716-4f46-8eae-1b7cd0f2294d PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:32:43.609Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55620cce-809b-4fed-ab0d-3f56aad7e26e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:32:43.609Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55620cce-809b-4fed-ab0d-3f56aad7e26e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@289f5bf0 2021-08-11T16:32:43.609Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55620cce-809b-4fed-ab0d-3f56aad7e26e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:32:43.609Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55620cce-809b-4fed-ab0d-3f56aad7e26e PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:32:46.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11fd46da-db05-499b-9184-b3371a21f726 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:32:46.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11fd46da-db05-499b-9184-b3371a21f726 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1677dbd5 2021-08-11T16:32:46.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11fd46da-db05-499b-9184-b3371a21f726 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:32:46.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11fd46da-db05-499b-9184-b3371a21f726 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:32:48.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:32:48.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:32:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:32:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:32:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:32:50.551Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c49a09e5-1fe0-4701-aef9-5f9a854c1b9e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:32:50.551Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c49a09e5-1fe0-4701-aef9-5f9a854c1b9e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@212726 2021-08-11T16:32:50.551Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c49a09e5-1fe0-4701-aef9-5f9a854c1b9e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:32:50.551Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c49a09e5-1fe0-4701-aef9-5f9a854c1b9e PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:32:53.455Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71d39a07-c092-4e53-8483-848ce8f28763 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:32:53.455Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71d39a07-c092-4e53-8483-848ce8f28763 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6e5927d2 2021-08-11T16:32:53.455Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71d39a07-c092-4e53-8483-848ce8f28763 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:32:53.456Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71d39a07-c092-4e53-8483-848ce8f28763 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:32:56.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ff8f2882-6296-4f40-8d5f-3d8bc7903964 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:32:56.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ff8f2882-6296-4f40-8d5f-3d8bc7903964 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3c4ba132 2021-08-11T16:32:56.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ff8f2882-6296-4f40-8d5f-3d8bc7903964 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:32:56.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ff8f2882-6296-4f40-8d5f-3d8bc7903964 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:33:00.046Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=62298810-e6ff-42c3-8d32-3650ce6d7ee9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:33:00.046Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=62298810-e6ff-42c3-8d32-3650ce6d7ee9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@144a6c1d 2021-08-11T16:33:00.046Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=62298810-e6ff-42c3-8d32-3650ce6d7ee9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:33:00.046Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=62298810-e6ff-42c3-8d32-3650ce6d7ee9 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:33:02.055Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58d05729-73d5-4a31-8ce7-2df91410c7b7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:33:02.055Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58d05729-73d5-4a31-8ce7-2df91410c7b7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1d5b61a5 2021-08-11T16:33:02.055Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58d05729-73d5-4a31-8ce7-2df91410c7b7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:33:02.055Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58d05729-73d5-4a31-8ce7-2df91410c7b7 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:33:03.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:33:03.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:33:04.251Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:33:04.617Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86e6eccc-75b6-450d-873e-1e3278e6b895 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:33:04.617Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86e6eccc-75b6-450d-873e-1e3278e6b895 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3a1bb26b 2021-08-11T16:33:04.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86e6eccc-75b6-450d-873e-1e3278e6b895 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:33:04.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86e6eccc-75b6-450d-873e-1e3278e6b895 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:33:07.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4832252a-0db3-4972-a754-1399822e8a83 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:33:07.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4832252a-0db3-4972-a754-1399822e8a83 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5bb8c62b 2021-08-11T16:33:07.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4832252a-0db3-4972-a754-1399822e8a83 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:33:07.608Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4832252a-0db3-4972-a754-1399822e8a83 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:33:10.594Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1217b574-b152-4d3f-b1ac-1b9111591e06 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:33:10.594Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1217b574-b152-4d3f-b1ac-1b9111591e06 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1cd573c5 2021-08-11T16:33:10.595Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1217b574-b152-4d3f-b1ac-1b9111591e06 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:33:10.595Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1217b574-b152-4d3f-b1ac-1b9111591e06 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:33:13.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a792e0c-14b9-43e6-8aa7-1a409afd4d01 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:33:13.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a792e0c-14b9-43e6-8aa7-1a409afd4d01 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5a457fb3 2021-08-11T16:33:13.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a792e0c-14b9-43e6-8aa7-1a409afd4d01 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:33:13.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a792e0c-14b9-43e6-8aa7-1a409afd4d01 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:33:16.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a485abe-66f0-4cc9-9da7-ee594ff089f4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:33:16.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a485abe-66f0-4cc9-9da7-ee594ff089f4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3d2a167a 2021-08-11T16:33:16.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a485abe-66f0-4cc9-9da7-ee594ff089f4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:33:16.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a485abe-66f0-4cc9-9da7-ee594ff089f4 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:33:18.310Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:33:18.323Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:33:19.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8a41199-f269-42ea-b9fc-f1c01fbdb26e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:33:19.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8a41199-f269-42ea-b9fc-f1c01fbdb26e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@20be166a 2021-08-11T16:33:19.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8a41199-f269-42ea-b9fc-f1c01fbdb26e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:33:19.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8a41199-f269-42ea-b9fc-f1c01fbdb26e PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:33:22.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=45bb312e-48cd-4662-ab9f-f18301e6a7dc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:33:22.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=45bb312e-48cd-4662-ab9f-f18301e6a7dc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@17c212e3 2021-08-11T16:33:22.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=45bb312e-48cd-4662-ab9f-f18301e6a7dc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:33:22.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=45bb312e-48cd-4662-ab9f-f18301e6a7dc PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:33:25.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a0dabc1-183d-4989-a892-e08d8fe93837 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:33:25.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a0dabc1-183d-4989-a892-e08d8fe93837 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@660b5f4f 2021-08-11T16:33:25.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a0dabc1-183d-4989-a892-e08d8fe93837 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:33:25.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4a0dabc1-183d-4989-a892-e08d8fe93837 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:33:28.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55795e1c-f09e-4319-b000-57bd8288f294 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:33:28.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55795e1c-f09e-4319-b000-57bd8288f294 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@b7f1c32 2021-08-11T16:33:28.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55795e1c-f09e-4319-b000-57bd8288f294 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:33:28.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55795e1c-f09e-4319-b000-57bd8288f294 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:33:31.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01e20aed-aa24-449e-9c69-981707882d96 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:33:31.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01e20aed-aa24-449e-9c69-981707882d96 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5b225671 2021-08-11T16:33:31.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01e20aed-aa24-449e-9c69-981707882d96 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:33:31.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01e20aed-aa24-449e-9c69-981707882d96 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:33:33.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:33:33.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:33:34.710Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40230d34-ce4f-4203-901c-74c051d0f747 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:33:34.711Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40230d34-ce4f-4203-901c-74c051d0f747 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5cf002fc 2021-08-11T16:33:34.711Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40230d34-ce4f-4203-901c-74c051d0f747 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:33:34.711Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=40230d34-ce4f-4203-901c-74c051d0f747 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:33:37.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0ba53cc-5c83-4ea1-9da4-91380f4f9256 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:33:37.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0ba53cc-5c83-4ea1-9da4-91380f4f9256 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@60bdcf0a 2021-08-11T16:33:37.776Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0ba53cc-5c83-4ea1-9da4-91380f4f9256 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:33:37.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0ba53cc-5c83-4ea1-9da4-91380f4f9256 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:33:40.850Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=895f1156-1975-4a4e-b06d-305f9a62e606 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:33:40.850Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=895f1156-1975-4a4e-b06d-305f9a62e606 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@13d0d1a3 2021-08-11T16:33:40.850Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=895f1156-1975-4a4e-b06d-305f9a62e606 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:33:40.851Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=895f1156-1975-4a4e-b06d-305f9a62e606 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:33:43.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=89f204a0-206a-4287-9023-545cc69d5c3c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:33:43.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=89f204a0-206a-4287-9023-545cc69d5c3c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5a088c76 2021-08-11T16:33:43.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=89f204a0-206a-4287-9023-545cc69d5c3c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:33:43.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=89f204a0-206a-4287-9023-545cc69d5c3c PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:33:46.605Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aec0d214-08bc-457a-818f-268396d20041 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:33:46.605Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aec0d214-08bc-457a-818f-268396d20041 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6b39229 2021-08-11T16:33:46.605Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aec0d214-08bc-457a-818f-268396d20041 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:33:46.605Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aec0d214-08bc-457a-818f-268396d20041 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:33:48.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:33:48.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:33:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:33:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:33:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:33:49.951Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=726d7950-f0fa-4c4a-b493-54bcc39ebb86 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:33:49.951Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=726d7950-f0fa-4c4a-b493-54bcc39ebb86 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@8f8d86a 2021-08-11T16:33:49.951Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=726d7950-f0fa-4c4a-b493-54bcc39ebb86 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:33:49.951Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=726d7950-f0fa-4c4a-b493-54bcc39ebb86 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:33:53.452Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee4e0ebf-b27e-402f-a748-1db10438e097 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:33:53.452Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee4e0ebf-b27e-402f-a748-1db10438e097 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1db04d1f 2021-08-11T16:33:53.452Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee4e0ebf-b27e-402f-a748-1db10438e097 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:33:53.452Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee4e0ebf-b27e-402f-a748-1db10438e097 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:33:55.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b51311db-3733-466b-96da-117e5f54bbc4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:33:55.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b51311db-3733-466b-96da-117e5f54bbc4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@34519e3 2021-08-11T16:33:55.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b51311db-3733-466b-96da-117e5f54bbc4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:33:55.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b51311db-3733-466b-96da-117e5f54bbc4 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:33:58.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd5dd84d-ac09-477e-8521-a372e2d899d5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:33:58.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd5dd84d-ac09-477e-8521-a372e2d899d5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@197df178 2021-08-11T16:33:58.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd5dd84d-ac09-477e-8521-a372e2d899d5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:33:58.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd5dd84d-ac09-477e-8521-a372e2d899d5 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:34:01.652Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1cc5d617-660f-40b2-9c53-3f5e592b8941 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:34:01.652Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1cc5d617-660f-40b2-9c53-3f5e592b8941 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@578851b5 2021-08-11T16:34:01.652Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1cc5d617-660f-40b2-9c53-3f5e592b8941 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:34:01.652Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1cc5d617-660f-40b2-9c53-3f5e592b8941 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:34:03.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:34:03.311Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:34:04.261Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:34:04.608Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4395003b-aa91-46cd-9b3d-0e3f8728049b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:34:04.608Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4395003b-aa91-46cd-9b3d-0e3f8728049b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@48cf36ee 2021-08-11T16:34:04.608Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4395003b-aa91-46cd-9b3d-0e3f8728049b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:34:04.608Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4395003b-aa91-46cd-9b3d-0e3f8728049b PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:34:07.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=720e0bc4-3fef-45c0-9f53-b56ff8c02325 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:34:07.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=720e0bc4-3fef-45c0-9f53-b56ff8c02325 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@63eea415 2021-08-11T16:34:07.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=720e0bc4-3fef-45c0-9f53-b56ff8c02325 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:34:07.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=720e0bc4-3fef-45c0-9f53-b56ff8c02325 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:34:10.605Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b79b36e2-31e3-44b1-a089-4083abf265b7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:34:10.605Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b79b36e2-31e3-44b1-a089-4083abf265b7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@156e6f21 2021-08-11T16:34:10.605Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b79b36e2-31e3-44b1-a089-4083abf265b7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:34:10.605Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b79b36e2-31e3-44b1-a089-4083abf265b7 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:34:13.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9c4b1336-7d6b-4bba-9370-c468b94c6bae PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:34:13.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9c4b1336-7d6b-4bba-9370-c468b94c6bae PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7e91c0 2021-08-11T16:34:13.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9c4b1336-7d6b-4bba-9370-c468b94c6bae PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:34:13.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9c4b1336-7d6b-4bba-9370-c468b94c6bae PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:34:16.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=130a5fab-86b8-42ce-a182-8dc2ec3ea5dd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:34:16.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=130a5fab-86b8-42ce-a182-8dc2ec3ea5dd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@31b7af67 2021-08-11T16:34:16.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=130a5fab-86b8-42ce-a182-8dc2ec3ea5dd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:34:16.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=130a5fab-86b8-42ce-a182-8dc2ec3ea5dd PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:34:18.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:34:18.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:34:19.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75fab609-eeb5-44ed-9267-469d4bde9f24 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:34:19.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75fab609-eeb5-44ed-9267-469d4bde9f24 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@276f7b48 2021-08-11T16:34:19.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75fab609-eeb5-44ed-9267-469d4bde9f24 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:34:19.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=75fab609-eeb5-44ed-9267-469d4bde9f24 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:34:22.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=da9e239e-ff8d-46cd-86b8-1fd0e22c4388 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:34:22.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=da9e239e-ff8d-46cd-86b8-1fd0e22c4388 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@411e1fe3 2021-08-11T16:34:22.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=da9e239e-ff8d-46cd-86b8-1fd0e22c4388 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:34:22.617Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=da9e239e-ff8d-46cd-86b8-1fd0e22c4388 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:34:25.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70dea860-b2e8-4b3b-9695-c9c5b3013f18 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:34:25.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70dea860-b2e8-4b3b-9695-c9c5b3013f18 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@351a6691 2021-08-11T16:34:25.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70dea860-b2e8-4b3b-9695-c9c5b3013f18 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:34:25.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70dea860-b2e8-4b3b-9695-c9c5b3013f18 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:34:28.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21300c4b-f0c8-4b75-a04a-2dd6fe0ac644 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:34:28.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21300c4b-f0c8-4b75-a04a-2dd6fe0ac644 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@39da3bc2 2021-08-11T16:34:28.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21300c4b-f0c8-4b75-a04a-2dd6fe0ac644 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:34:28.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21300c4b-f0c8-4b75-a04a-2dd6fe0ac644 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:34:31.612Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06ce2ca4-39f1-4cc9-a95e-247a47f1f7f2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:34:31.613Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06ce2ca4-39f1-4cc9-a95e-247a47f1f7f2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@65b3b28a 2021-08-11T16:34:31.613Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06ce2ca4-39f1-4cc9-a95e-247a47f1f7f2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:34:31.613Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06ce2ca4-39f1-4cc9-a95e-247a47f1f7f2 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:34:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:34:33.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:34:34.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0635f74c-8aa8-4cbf-b4b2-abe5f2096897 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:34:34.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0635f74c-8aa8-4cbf-b4b2-abe5f2096897 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@546348d8 2021-08-11T16:34:34.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0635f74c-8aa8-4cbf-b4b2-abe5f2096897 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:34:34.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0635f74c-8aa8-4cbf-b4b2-abe5f2096897 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:34:37.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c5c1eab-bb82-4d19-ac6a-0dd02f8700b4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:34:37.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c5c1eab-bb82-4d19-ac6a-0dd02f8700b4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@32ab922b 2021-08-11T16:34:37.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c5c1eab-bb82-4d19-ac6a-0dd02f8700b4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:34:37.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c5c1eab-bb82-4d19-ac6a-0dd02f8700b4 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:34:40.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bcb7c24b-3c19-4bef-ac8c-633e1c49f293 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:34:40.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bcb7c24b-3c19-4bef-ac8c-633e1c49f293 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5a47bb97 2021-08-11T16:34:40.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bcb7c24b-3c19-4bef-ac8c-633e1c49f293 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:34:40.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bcb7c24b-3c19-4bef-ac8c-633e1c49f293 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:34:43.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b9cad771-48a3-4472-b77e-ac105568a591 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:34:43.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b9cad771-48a3-4472-b77e-ac105568a591 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@221865b7 2021-08-11T16:34:43.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b9cad771-48a3-4472-b77e-ac105568a591 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:34:43.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b9cad771-48a3-4472-b77e-ac105568a591 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:34:46.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1f3b1a26-7f05-4143-915a-6b7ac0884a84 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:34:46.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1f3b1a26-7f05-4143-915a-6b7ac0884a84 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6cf3dca4 2021-08-11T16:34:46.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1f3b1a26-7f05-4143-915a-6b7ac0884a84 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:34:46.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1f3b1a26-7f05-4143-915a-6b7ac0884a84 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:34:48.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:34:48.317Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:34:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:34:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:34:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:34:50.450Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90e06a2e-85cc-43a3-80b9-8b0f6a777222 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:34:50.450Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90e06a2e-85cc-43a3-80b9-8b0f6a777222 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2a721db 2021-08-11T16:34:50.450Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90e06a2e-85cc-43a3-80b9-8b0f6a777222 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:34:50.450Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90e06a2e-85cc-43a3-80b9-8b0f6a777222 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:34:53.055Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c161885-3f1e-4b1e-b223-d04a4ef666ae PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:34:53.055Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c161885-3f1e-4b1e-b223-d04a4ef666ae PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@65dbb91b 2021-08-11T16:34:53.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c161885-3f1e-4b1e-b223-d04a4ef666ae PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:34:53.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c161885-3f1e-4b1e-b223-d04a4ef666ae PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:34:56.850Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eeca6681-5631-40e0-aa6c-aa50091d0fb1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:34:56.850Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eeca6681-5631-40e0-aa6c-aa50091d0fb1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@48f4dca 2021-08-11T16:34:56.850Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eeca6681-5631-40e0-aa6c-aa50091d0fb1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:34:56.850Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eeca6681-5631-40e0-aa6c-aa50091d0fb1 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:34:59.750Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=780c2ca8-5b34-42e3-bd0a-35c90eb67ceb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:34:59.750Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=780c2ca8-5b34-42e3-bd0a-35c90eb67ceb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@766bb703 2021-08-11T16:34:59.750Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=780c2ca8-5b34-42e3-bd0a-35c90eb67ceb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:34:59.750Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=780c2ca8-5b34-42e3-bd0a-35c90eb67ceb PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:35:01.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0a7802c6-f382-4514-b599-e07138051068 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:35:01.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0a7802c6-f382-4514-b599-e07138051068 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4df9e9ec 2021-08-11T16:35:01.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0a7802c6-f382-4514-b599-e07138051068 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:35:01.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0a7802c6-f382-4514-b599-e07138051068 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:35:03.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:35:03.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:35:04.248Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:35:04.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c80a84dc-1f44-452b-a66c-5dc4375738fd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:35:04.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c80a84dc-1f44-452b-a66c-5dc4375738fd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@75d29229 2021-08-11T16:35:04.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c80a84dc-1f44-452b-a66c-5dc4375738fd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:35:04.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c80a84dc-1f44-452b-a66c-5dc4375738fd PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:35:07.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01be6055-be2d-44ed-9228-799ab0d9639d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:35:07.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01be6055-be2d-44ed-9228-799ab0d9639d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@bcd6044 2021-08-11T16:35:07.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01be6055-be2d-44ed-9228-799ab0d9639d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:35:07.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01be6055-be2d-44ed-9228-799ab0d9639d PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:35:10.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5a0fa88-9b59-474a-9b5f-d6cfe9bf6671 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:35:10.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5a0fa88-9b59-474a-9b5f-d6cfe9bf6671 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@20350d91 2021-08-11T16:35:10.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5a0fa88-9b59-474a-9b5f-d6cfe9bf6671 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:35:10.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5a0fa88-9b59-474a-9b5f-d6cfe9bf6671 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:35:13.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a2ab1e74-d0d2-49c0-878b-a8a7c739f513 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:35:13.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a2ab1e74-d0d2-49c0-878b-a8a7c739f513 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@220126f2 2021-08-11T16:35:13.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a2ab1e74-d0d2-49c0-878b-a8a7c739f513 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:35:13.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a2ab1e74-d0d2-49c0-878b-a8a7c739f513 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:35:16.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0df6cfe-24e3-4e15-a104-dc28325a0d01 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:35:16.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0df6cfe-24e3-4e15-a104-dc28325a0d01 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@43a3bb41 2021-08-11T16:35:16.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0df6cfe-24e3-4e15-a104-dc28325a0d01 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:35:16.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0df6cfe-24e3-4e15-a104-dc28325a0d01 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:35:18.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:35:18.316Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:35:19.606Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f9fbe5b8-4de9-40c2-bed0-8f0efe4f8288 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:35:19.606Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f9fbe5b8-4de9-40c2-bed0-8f0efe4f8288 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@238c2c31 2021-08-11T16:35:19.606Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f9fbe5b8-4de9-40c2-bed0-8f0efe4f8288 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:35:19.606Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f9fbe5b8-4de9-40c2-bed0-8f0efe4f8288 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:35:22.652Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf9aab0c-b09f-45d3-8de7-4b16e0bbcffb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:35:22.652Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf9aab0c-b09f-45d3-8de7-4b16e0bbcffb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2bddf454 2021-08-11T16:35:22.652Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf9aab0c-b09f-45d3-8de7-4b16e0bbcffb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:35:22.652Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf9aab0c-b09f-45d3-8de7-4b16e0bbcffb PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:35:25.687Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=62fd17fd-6234-4544-8abf-20471809c54e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:35:25.687Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=62fd17fd-6234-4544-8abf-20471809c54e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@c0716b 2021-08-11T16:35:25.687Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=62fd17fd-6234-4544-8abf-20471809c54e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:35:25.687Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=62fd17fd-6234-4544-8abf-20471809c54e PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:35:28.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef54f672-6698-4f72-a4de-759bffeedbff PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:35:28.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef54f672-6698-4f72-a4de-759bffeedbff PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5e67bb81 2021-08-11T16:35:28.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef54f672-6698-4f72-a4de-759bffeedbff PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:35:28.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef54f672-6698-4f72-a4de-759bffeedbff PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:35:29.308Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0fbshgx1zwlbi1sbiyzeh1b5zo11 2021-08-11T16:35:29.309Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0p1ipxv70izu11oy5j21ureym417 2021-08-11T16:35:29.310Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0a3w6h1zt8cm7569i0c76d4ig5 2021-08-11T16:35:29.310Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node018g6h08h6j5it4eeaesau3nld13 2021-08-11T16:35:29.310Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01bms4quehyqmonmzmlyk2sfmm2 2021-08-11T16:35:29.310Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01ppxmfxw7bl3e1c06dgudqvd3c3 2021-08-11T16:35:29.311Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01b50z6jdkms7746hj7js3fpq314 2021-08-11T16:35:29.311Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01l9gfr2ea0zeeafe5rk3ximiq19 2021-08-11T16:35:29.311Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0eisy4eo6aukp1nvplpsaj433y7 2021-08-11T16:35:29.311Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01uekmvbf66i5y3vv3z5g55clp0 2021-08-11T16:35:29.312Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01d3c3uwxwfycq1b86e1mjbqwm29 2021-08-11T16:35:29.312Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0yx6rqjyjkb2daaoctydrcexd8 2021-08-11T16:35:29.312Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0xnmuqfficpq3119vwc2j0hlqk12 2021-08-11T16:35:29.312Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0fgi105xlmac1ewez8kivk0x424 2021-08-11T16:35:29.312Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01hli30kumzjspdaci7fwja2fn23 2021-08-11T16:35:29.313Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0nviq78c4w6sz15umc9bktxrk221 2021-08-11T16:35:29.313Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0jjk3elondtiufaiyyz3tf4bg1 2021-08-11T16:35:29.313Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0qh4pwnw1cnzb1fxnb5tbzw7l918 2021-08-11T16:35:29.313Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0hb5wi25rbom5bt2kqu16uoyo20 2021-08-11T16:35:29.314Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node06q9eitvli2ju14hv6s3zxnyiq16 2021-08-11T16:35:29.314Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01oxistnzrb8cd18vlf8qqy8r3f22 2021-08-11T16:35:29.314Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01tltao45wiimv1gkij2robzj6c6 2021-08-11T16:35:29.314Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01b1upqqy4kjf11jlhswdfqxogy15 2021-08-11T16:35:29.314Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node015rryag7np9m7ca0uhevg5o2i4 2021-08-11T16:35:29.315Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01hw5t1z3cp0muoptettwz0adi10 2021-08-11T16:35:31.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3b43b7e-98f2-4ee3-81b8-08041a6f083c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:35:31.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3b43b7e-98f2-4ee3-81b8-08041a6f083c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1cc8e9d8 2021-08-11T16:35:31.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3b43b7e-98f2-4ee3-81b8-08041a6f083c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:35:31.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3b43b7e-98f2-4ee3-81b8-08041a6f083c PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:35:33.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:35:33.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:35:34.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa5fc9ca-3c37-4411-b531-bbe8894ef41b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:35:34.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa5fc9ca-3c37-4411-b531-bbe8894ef41b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5873d141 2021-08-11T16:35:34.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa5fc9ca-3c37-4411-b531-bbe8894ef41b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:35:34.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa5fc9ca-3c37-4411-b531-bbe8894ef41b PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:35:37.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ebe72fde-abc9-4098-a522-fb53e90051b7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:35:37.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ebe72fde-abc9-4098-a522-fb53e90051b7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5b3bb80d 2021-08-11T16:35:37.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ebe72fde-abc9-4098-a522-fb53e90051b7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:35:37.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ebe72fde-abc9-4098-a522-fb53e90051b7 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:35:40.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce9806af-b497-486d-86e9-ab1f6a50d2df PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:35:40.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce9806af-b497-486d-86e9-ab1f6a50d2df PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3863bb58 2021-08-11T16:35:40.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce9806af-b497-486d-86e9-ab1f6a50d2df PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:35:40.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce9806af-b497-486d-86e9-ab1f6a50d2df PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:35:43.604Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=194b8158-52ef-4741-9dc4-9500caf27a82 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:35:43.604Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=194b8158-52ef-4741-9dc4-9500caf27a82 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@633e03a5 2021-08-11T16:35:43.604Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=194b8158-52ef-4741-9dc4-9500caf27a82 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:35:43.604Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=194b8158-52ef-4741-9dc4-9500caf27a82 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:35:46.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ae38268-35f8-4e94-9137-1e2e1539bfe8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:35:46.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ae38268-35f8-4e94-9137-1e2e1539bfe8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@25012f5a 2021-08-11T16:35:46.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ae38268-35f8-4e94-9137-1e2e1539bfe8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:35:46.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9ae38268-35f8-4e94-9137-1e2e1539bfe8 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:35:48.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:35:48.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:35:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:35:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:35:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:35:49.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ba4ddd5-ef27-4bab-9637-371ca2a86685 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:35:49.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ba4ddd5-ef27-4bab-9637-371ca2a86685 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@237c1027 2021-08-11T16:35:49.977Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ba4ddd5-ef27-4bab-9637-371ca2a86685 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:35:49.978Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ba4ddd5-ef27-4bab-9637-371ca2a86685 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:35:53.148Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71545a3b-3442-4d9f-b1f9-7ac172205a3a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:35:53.148Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71545a3b-3442-4d9f-b1f9-7ac172205a3a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@660d66b 2021-08-11T16:35:53.148Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71545a3b-3442-4d9f-b1f9-7ac172205a3a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:35:53.148Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71545a3b-3442-4d9f-b1f9-7ac172205a3a PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:35:55.600Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e31b0f0-f569-4959-9d0f-6e33ecac948e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:35:55.600Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e31b0f0-f569-4959-9d0f-6e33ecac948e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@15d0ce29 2021-08-11T16:35:55.600Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e31b0f0-f569-4959-9d0f-6e33ecac948e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:35:55.600Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e31b0f0-f569-4959-9d0f-6e33ecac948e PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:35:58.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47d7c21e-8058-4915-a875-db3671a189c5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:35:58.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47d7c21e-8058-4915-a875-db3671a189c5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@39074836 2021-08-11T16:35:58.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47d7c21e-8058-4915-a875-db3671a189c5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:35:58.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47d7c21e-8058-4915-a875-db3671a189c5 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:36:01.602Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5873b0d4-7301-4724-b109-d02ed3d1eaa1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:36:01.602Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5873b0d4-7301-4724-b109-d02ed3d1eaa1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7a398a19 2021-08-11T16:36:01.602Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5873b0d4-7301-4724-b109-d02ed3d1eaa1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:36:01.602Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5873b0d4-7301-4724-b109-d02ed3d1eaa1 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:36:03.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:36:03.310Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:36:04.253Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:36:04.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=57459b75-65a9-418a-85f5-e74ad8b894c6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:36:04.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=57459b75-65a9-418a-85f5-e74ad8b894c6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7a6ad8bb 2021-08-11T16:36:04.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=57459b75-65a9-418a-85f5-e74ad8b894c6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:36:04.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=57459b75-65a9-418a-85f5-e74ad8b894c6 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:36:07.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d1a9883-65b4-4273-8d35-94d79830ee54 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:36:07.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d1a9883-65b4-4273-8d35-94d79830ee54 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6958e059 2021-08-11T16:36:07.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d1a9883-65b4-4273-8d35-94d79830ee54 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:36:07.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d1a9883-65b4-4273-8d35-94d79830ee54 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:36:10.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=33de8b11-873a-4dfa-8893-1cb757506bb5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:36:10.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=33de8b11-873a-4dfa-8893-1cb757506bb5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6a9d53bc 2021-08-11T16:36:10.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=33de8b11-873a-4dfa-8893-1cb757506bb5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:36:10.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=33de8b11-873a-4dfa-8893-1cb757506bb5 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:36:13.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7bd9ff3-4907-4a86-ba3d-3617950d8573 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:36:13.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7bd9ff3-4907-4a86-ba3d-3617950d8573 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5c4265b4 2021-08-11T16:36:13.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7bd9ff3-4907-4a86-ba3d-3617950d8573 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:36:13.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c7bd9ff3-4907-4a86-ba3d-3617950d8573 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:36:16.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec658233-6cb3-4345-be88-feedd2fb573f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:36:16.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec658233-6cb3-4345-be88-feedd2fb573f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7041c187 2021-08-11T16:36:16.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec658233-6cb3-4345-be88-feedd2fb573f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:36:16.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec658233-6cb3-4345-be88-feedd2fb573f PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:36:18.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:36:18.317Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:36:19.608Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=efae6b46-630f-48c7-a350-bdf62d4ebdd3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:36:19.608Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=efae6b46-630f-48c7-a350-bdf62d4ebdd3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5eb5adfe 2021-08-11T16:36:19.608Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=efae6b46-630f-48c7-a350-bdf62d4ebdd3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:36:19.608Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=efae6b46-630f-48c7-a350-bdf62d4ebdd3 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:36:22.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c283cdc3-a296-4d00-a370-4e0e4d3f79e5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:36:22.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c283cdc3-a296-4d00-a370-4e0e4d3f79e5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5eb51527 2021-08-11T16:36:22.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c283cdc3-a296-4d00-a370-4e0e4d3f79e5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:36:22.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c283cdc3-a296-4d00-a370-4e0e4d3f79e5 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:36:25.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0d466632-c030-4c3c-bcb5-839745dfe80f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:36:25.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0d466632-c030-4c3c-bcb5-839745dfe80f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1195b480 2021-08-11T16:36:25.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0d466632-c030-4c3c-bcb5-839745dfe80f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:36:25.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0d466632-c030-4c3c-bcb5-839745dfe80f PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:36:28.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef2a2a54-17bd-4add-a800-58d1f90ef3bd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:36:28.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef2a2a54-17bd-4add-a800-58d1f90ef3bd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6f2602cc 2021-08-11T16:36:28.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef2a2a54-17bd-4add-a800-58d1f90ef3bd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:36:28.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef2a2a54-17bd-4add-a800-58d1f90ef3bd PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:36:31.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e17a397-660c-43f7-b7c3-9af0130af713 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:36:31.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e17a397-660c-43f7-b7c3-9af0130af713 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1c0f988b 2021-08-11T16:36:31.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e17a397-660c-43f7-b7c3-9af0130af713 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:36:31.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e17a397-660c-43f7-b7c3-9af0130af713 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:36:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:36:33.316Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:36:34.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0dd980a7-2c47-4ef6-b0f4-adafacf1869e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:36:34.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0dd980a7-2c47-4ef6-b0f4-adafacf1869e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6ae5aa00 2021-08-11T16:36:34.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0dd980a7-2c47-4ef6-b0f4-adafacf1869e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:36:34.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0dd980a7-2c47-4ef6-b0f4-adafacf1869e PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:36:37.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=430dd3af-323c-40a6-a4fe-41a349b0c3fe PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:36:37.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=430dd3af-323c-40a6-a4fe-41a349b0c3fe PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@56683268 2021-08-11T16:36:37.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=430dd3af-323c-40a6-a4fe-41a349b0c3fe PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:36:37.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=430dd3af-323c-40a6-a4fe-41a349b0c3fe PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:36:40.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a6853336-4770-4efb-9fae-30d158ee05c0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:36:40.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a6853336-4770-4efb-9fae-30d158ee05c0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@9876a0b 2021-08-11T16:36:40.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a6853336-4770-4efb-9fae-30d158ee05c0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:36:40.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a6853336-4770-4efb-9fae-30d158ee05c0 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:36:43.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d96698f7-3639-4458-b4db-c553e4d16bc6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:36:43.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d96698f7-3639-4458-b4db-c553e4d16bc6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@692c7070 2021-08-11T16:36:43.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d96698f7-3639-4458-b4db-c553e4d16bc6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:36:43.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d96698f7-3639-4458-b4db-c553e4d16bc6 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:36:46.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=838b0b76-da21-4855-ae48-d0e7ec4bb45d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:36:46.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=838b0b76-da21-4855-ae48-d0e7ec4bb45d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2e5e72ff 2021-08-11T16:36:46.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=838b0b76-da21-4855-ae48-d0e7ec4bb45d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:36:46.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=838b0b76-da21-4855-ae48-d0e7ec4bb45d PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:36:48.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:36:48.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:36:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:36:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:36:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:36:50.061Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b3520d1-37e5-4f40-847c-eec041b77e37 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:36:50.061Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b3520d1-37e5-4f40-847c-eec041b77e37 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@39b2fe2e 2021-08-11T16:36:50.061Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b3520d1-37e5-4f40-847c-eec041b77e37 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:36:50.061Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b3520d1-37e5-4f40-847c-eec041b77e37 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:36:53.448Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=844decb0-4621-4110-89b1-307536521be8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:36:53.448Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=844decb0-4621-4110-89b1-307536521be8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@bd71c21 2021-08-11T16:36:53.448Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=844decb0-4621-4110-89b1-307536521be8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:36:53.448Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=844decb0-4621-4110-89b1-307536521be8 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:36:56.550Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5380ed5-21a2-48ee-b8a9-75d944a03bd0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:36:56.550Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5380ed5-21a2-48ee-b8a9-75d944a03bd0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1f252db 2021-08-11T16:36:56.550Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5380ed5-21a2-48ee-b8a9-75d944a03bd0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:36:56.550Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5380ed5-21a2-48ee-b8a9-75d944a03bd0 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:36:59.357Z [cluster1-nio-worker-1] WARN c.d.driver.core.HostConnectionPool Error creating connection to cassandra.onap/10.242.184.131:9042 com.datastax.driver.core.exceptions.TransportException: [cassandra.onap/10.242.184.131:9042] Connection has been closed at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1427) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1408) at com.datastax.driver.core.Connection.defunct(Connection.java:585) at com.datastax.driver.core.Connection$Dispatcher.exceptionCaught(Connection.java:1330) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273) at io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.handleReadException(AbstractEpollStreamChannel.java:728) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:818) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-08-11T16:36:59.358Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.184.131:9042 com.datastax.driver.core.exceptions.ConnectionException: [cassandra.onap/10.242.184.131:9042] Pool was closed during initialization at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:173) at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:158) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1080) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.google.common.util.concurrent.CollectionFuture.handleAllCompleted(CollectionFuture.java:69) at com.google.common.util.concurrent.AggregateFuture.processCompleted(AggregateFuture.java:282) at com.google.common.util.concurrent.AggregateFuture.decrementCountAndMaybeComplete(AggregateFuture.java:264) at com.google.common.util.concurrent.AggregateFuture.access$200(AggregateFuture.java:42) at com.google.common.util.concurrent.AggregateFuture$1.run(AggregateFuture.java:147) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:100) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.datastax.driver.core.Connection$Future.onException(Connection.java:1518) at com.datastax.driver.core.Connection$Dispatcher.errorOutAllHandler(Connection.java:1340) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1427) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1408) at com.datastax.driver.core.Connection.defunct(Connection.java:585) at com.datastax.driver.core.Connection$Dispatcher.exceptionCaught(Connection.java:1330) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273) at io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.handleReadException(AbstractEpollStreamChannel.java:728) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:818) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-08-11T16:36:59.358Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a8bf8ede-47a5-4700-a326-f22c7dbfd189 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:36:59.358Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a8bf8ede-47a5-4700-a326-f22c7dbfd189 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5ce61190 2021-08-11T16:36:59.358Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a8bf8ede-47a5-4700-a326-f22c7dbfd189 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:36:59.358Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a8bf8ede-47a5-4700-a326-f22c7dbfd189 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:36:59.866Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=a8bf8ede-47a5-4700-a326-f22c7dbfd189 ErrorCategory=INFO ServiceName=BE Component CASSANDRA state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component CASSANDRA state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:36:59.866Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=a8bf8ede-47a5-4700-a326-f22c7dbfd189 ErrorCategory=ERROR ServiceName=BE Component CASSANDRA state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-08-11T16:37:01.587Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85bfa1d5-0625-4702-9bf3-08dd68e092b2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:37:01.587Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85bfa1d5-0625-4702-9bf3-08dd68e092b2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@720bc7ac 2021-08-11T16:37:01.587Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85bfa1d5-0625-4702-9bf3-08dd68e092b2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:37:01.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85bfa1d5-0625-4702-9bf3-08dd68e092b2 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:37:02.249Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:02.276Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:02.328Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:02.431Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:02.633Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:03.037Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:03.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:37:03.245Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-08-11T16:37:03.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:37:03.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:37:03.440Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:04.244Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:04.253Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:37:04.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=927d3a5d-cbac-4346-9a60-0060be8fdff3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:37:04.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=927d3a5d-cbac-4346-9a60-0060be8fdff3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@410eebd8 2021-08-11T16:37:04.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=927d3a5d-cbac-4346-9a60-0060be8fdff3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:37:04.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=927d3a5d-cbac-4346-9a60-0060be8fdff3 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:37:04.601Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=927d3a5d-cbac-4346-9a60-0060be8fdff3 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:37:04.602Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=927d3a5d-cbac-4346-9a60-0060be8fdff3 ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-08-11T16:37:05.047Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:05.168Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:05.196Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:05.248Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:05.301Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:05.404Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:05.607Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:05.809Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:06.211Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:06.245Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:37:06.614Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:06.649Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:07.017Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:07.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3dc64cd7-dcb2-48b1-b19a-ea9a8a4a109f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:37:07.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3dc64cd7-dcb2-48b1-b19a-ea9a8a4a109f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@52f90da8 2021-08-11T16:37:07.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3dc64cd7-dcb2-48b1-b19a-ea9a8a4a109f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:37:07.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3dc64cd7-dcb2-48b1-b19a-ea9a8a4a109f PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:37:07.820Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T16:36:59.661799Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:09.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:37:09.853Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:09.880Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:09.907Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:09.959Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:10.062Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:10.164Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:10.271Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:527) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:10.473Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:10.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f62abbd7-0647-4d5f-94ee-118b995deb67 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:37:10.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f62abbd7-0647-4d5f-94ee-118b995deb67 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3e7fecf6 2021-08-11T16:37:10.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f62abbd7-0647-4d5f-94ee-118b995deb67 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:37:10.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f62abbd7-0647-4d5f-94ee-118b995deb67 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:37:10.676Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:11.078Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:11.881Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:12.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:37:12.823Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:12.876Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:12.929Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:12.995Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:13.049Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:13.147Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:13.265Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:13.367Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:13.484Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:13.572Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:13.596Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb5a2a35-6017-41d0-b617-e138fda35739 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:37:13.596Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb5a2a35-6017-41d0-b617-e138fda35739 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@12657a32 2021-08-11T16:37:13.596Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb5a2a35-6017-41d0-b617-e138fda35739 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:37:13.596Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb5a2a35-6017-41d0-b617-e138fda35739 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:37:13.775Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:13.977Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:14.379Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:14.782Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:15.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:37:15.585Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:16.387Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T16:36:59.661799Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:16.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a5db835c-d2c1-4391-ad19-a728291384a3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:37:16.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a5db835c-d2c1-4391-ad19-a728291384a3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2403038d 2021-08-11T16:37:16.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a5db835c-d2c1-4391-ad19-a728291384a3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:37:16.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a5db835c-d2c1-4391-ad19-a728291384a3 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:37:16.688Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:16.715Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:16.742Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:16.794Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:16.846Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:16.948Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:17.151Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:17.554Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:17.956Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:18.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:37:18.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:37:18.316Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:37:18.359Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:18.762Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:19.165Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:19.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fda19e04-5848-4bd3-83d4-f5dc0496ca72 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:37:19.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fda19e04-5848-4bd3-83d4-f5dc0496ca72 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3e8ddee8 2021-08-11T16:37:19.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fda19e04-5848-4bd3-83d4-f5dc0496ca72 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:37:19.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fda19e04-5848-4bd3-83d4-f5dc0496ca72 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:37:19.968Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:21.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:37:21.390Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:21.417Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:21.445Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:21.472Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:21.525Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:21.571Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:527) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:21.627Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:21.730Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:21.833Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:22.036Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:22.438Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:22.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f7262da-e49b-4e25-9495-1ce8d959c1e0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:37:22.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f7262da-e49b-4e25-9495-1ce8d959c1e0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7f549d93 2021-08-11T16:37:22.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f7262da-e49b-4e25-9495-1ce8d959c1e0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:37:22.594Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f7262da-e49b-4e25-9495-1ce8d959c1e0 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:37:22.841Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:23.174Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:23.244Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:23.646Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:24.050Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:24.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:37:24.453Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:25.256Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T16:36:59.661799Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:25.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8e710d43-6e8f-4938-a52f-9c50e0c290ca PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:37:25.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8e710d43-6e8f-4938-a52f-9c50e0c290ca PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@46d9dca1 2021-08-11T16:37:25.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8e710d43-6e8f-4938-a52f-9c50e0c290ca PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:37:25.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8e710d43-6e8f-4938-a52f-9c50e0c290ca PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:37:26.379Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:26.447Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:26.548Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:26.600Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:26.705Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:26.807Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:26.910Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:27.148Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:27.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:37:27.553Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:27.956Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:28.362Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:28.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=998818e3-d0fd-49a2-87dd-f2d53ae3eb2b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:37:28.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=998818e3-d0fd-49a2-87dd-f2d53ae3eb2b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7afab09c 2021-08-11T16:37:28.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=998818e3-d0fd-49a2-87dd-f2d53ae3eb2b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:37:28.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=998818e3-d0fd-49a2-87dd-f2d53ae3eb2b PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:37:29.165Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:30.246Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:37:30.259Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:30.286Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:30.339Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:30.442Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:30.645Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:30.776Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:30.847Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:31.250Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:31.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc479f51-cb17-43b3-8855-51102aec818f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:37:31.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc479f51-cb17-43b3-8855-51102aec818f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2ad04bb5 2021-08-11T16:37:31.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc479f51-cb17-43b3-8855-51102aec818f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:37:31.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc479f51-cb17-43b3-8855-51102aec818f PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:37:31.652Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:32.055Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:32.858Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:33.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:37:33.309Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:37:33.316Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:37:33.661Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T16:36:59.661799Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:33.980Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:34.007Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:34.059Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:34.111Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:34.163Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:34.215Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:34.268Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:34.320Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:34.372Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:34.425Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:34.527Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:34.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=afb83f1d-a62b-4a10-bcdb-e7e5659484ee PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:37:34.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=afb83f1d-a62b-4a10-bcdb-e7e5659484ee PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7133e379 2021-08-11T16:37:34.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=afb83f1d-a62b-4a10-bcdb-e7e5659484ee PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:37:34.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=afb83f1d-a62b-4a10-bcdb-e7e5659484ee PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:37:34.730Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:34.933Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:35.136Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:35.539Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:35.942Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:36.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:37:36.344Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:37.147Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:37.747Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ca626c83-176e-4c0c-a130-062f5c65f125 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:37:37.747Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ca626c83-176e-4c0c-a130-062f5c65f125 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7907152a 2021-08-11T16:37:37.747Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ca626c83-176e-4c0c-a130-062f5c65f125 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:37:37.747Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ca626c83-176e-4c0c-a130-062f5c65f125 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:37:38.665Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:38.748Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:38.749Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:38.775Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:38.849Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:38.951Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:39.153Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:39.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:37:39.556Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:39.958Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:40.761Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:40.950Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4bb81e15-3263-40bc-9426-98ea3dab84de PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:37:40.950Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4bb81e15-3263-40bc-9426-98ea3dab84de PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@565624c6 2021-08-11T16:37:40.950Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4bb81e15-3263-40bc-9426-98ea3dab84de PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:37:40.950Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4bb81e15-3263-40bc-9426-98ea3dab84de PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:37:41.564Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T16:36:59.661799Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:41.953Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:42.006Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:42.108Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:42.210Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:42.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:37:42.413Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:42.816Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:43.219Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:43.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2d4ed557-0399-4f77-9a6f-fcf708c086bd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:37:43.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2d4ed557-0399-4f77-9a6f-fcf708c086bd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7ef3505a 2021-08-11T16:37:43.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2d4ed557-0399-4f77-9a6f-fcf708c086bd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:37:43.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2d4ed557-0399-4f77-9a6f-fcf708c086bd PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:37:43.621Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:44.024Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:37:44.985Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-08-11T16:37:46.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ebb4cbe9-ae4a-4035-87a6-0851fc14b56e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:37:46.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ebb4cbe9-ae4a-4035-87a6-0851fc14b56e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@10b54cd6 2021-08-11T16:37:46.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ebb4cbe9-ae4a-4035-87a6-0851fc14b56e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:37:46.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ebb4cbe9-ae4a-4035-87a6-0851fc14b56e PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:37:46.605Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=ebb4cbe9-ae4a-4035-87a6-0851fc14b56e ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:37:46.605Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=ebb4cbe9-ae4a-4035-87a6-0851fc14b56e ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-08-11T16:37:48.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:37:48.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:37:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:37:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:37:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:37:49.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=abb4f425-aabc-4460-a4a7-484a418ab3c0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:37:49.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=abb4f425-aabc-4460-a4a7-484a418ab3c0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@51dab5af 2021-08-11T16:37:49.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=abb4f425-aabc-4460-a4a7-484a418ab3c0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:37:49.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=abb4f425-aabc-4460-a4a7-484a418ab3c0 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:37:50.357Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=abb4f425-aabc-4460-a4a7-484a418ab3c0 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:37:50.357Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=abb4f425-aabc-4460-a4a7-484a418ab3c0 ErrorCategory=ERROR ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-08-11T16:37:52.586Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4be7ca63-de26-4fbc-ac5c-4a0a781a6076 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:37:52.587Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4be7ca63-de26-4fbc-ac5c-4a0a781a6076 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@36cddb46 2021-08-11T16:37:52.587Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4be7ca63-de26-4fbc-ac5c-4a0a781a6076 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:37:52.587Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4be7ca63-de26-4fbc-ac5c-4a0a781a6076 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:37:55.601Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c46bd22-3b4e-4dd2-a59e-3a0e47a7b460 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:37:55.601Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c46bd22-3b4e-4dd2-a59e-3a0e47a7b460 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@52d82c45 2021-08-11T16:37:55.601Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c46bd22-3b4e-4dd2-a59e-3a0e47a7b460 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:37:55.601Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c46bd22-3b4e-4dd2-a59e-3a0e47a7b460 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:38:01.353Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c637637-3725-4b18-b933-dc554a321fb8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:38:01.353Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c637637-3725-4b18-b933-dc554a321fb8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@68310934 2021-08-11T16:38:01.353Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c637637-3725-4b18-b933-dc554a321fb8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:38:01.353Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c637637-3725-4b18-b933-dc554a321fb8 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:38:02.868Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=7c637637-3725-4b18-b933-dc554a321fb8 ErrorCategory=INFO ServiceName=BE Component CASSANDRA state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component CASSANDRA state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:38:02.868Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=7c637637-3725-4b18-b933-dc554a321fb8 ErrorCategory=INFO ServiceName=BE Component CASSANDRA state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:38:03.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:38:03.311Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:38:04.262Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:38:14.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef734b5c-4301-4914-ac50-529d78c4b0ef PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:38:14.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef734b5c-4301-4914-ac50-529d78c4b0ef PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6cc2be50 2021-08-11T16:38:14.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef734b5c-4301-4914-ac50-529d78c4b0ef PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:38:14.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef734b5c-4301-4914-ac50-529d78c4b0ef PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:38:17.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8bb8f0fd-c6c7-4563-aca4-adf5fbac17d0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:38:17.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8bb8f0fd-c6c7-4563-aca4-adf5fbac17d0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@348d5f65 2021-08-11T16:38:17.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8bb8f0fd-c6c7-4563-aca4-adf5fbac17d0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:38:17.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8bb8f0fd-c6c7-4563-aca4-adf5fbac17d0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:38:18.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:38:18.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:38:20.162Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=211aa7b2-32bc-4575-aeef-c29adb1ae2ab PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:38:20.162Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=211aa7b2-32bc-4575-aeef-c29adb1ae2ab PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@27a4f7b5 2021-08-11T16:38:20.162Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=211aa7b2-32bc-4575-aeef-c29adb1ae2ab PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:38:20.163Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=211aa7b2-32bc-4575-aeef-c29adb1ae2ab PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:38:22.074Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2db7f1e9-7c7e-42a5-b60a-0decdca80998 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:38:22.074Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2db7f1e9-7c7e-42a5-b60a-0decdca80998 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@473c22dc 2021-08-11T16:38:22.075Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2db7f1e9-7c7e-42a5-b60a-0decdca80998 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:38:22.075Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2db7f1e9-7c7e-42a5-b60a-0decdca80998 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:38:23.180Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8f2b7a1-50bc-4eb2-80e6-8c98d359efd1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:38:23.181Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8f2b7a1-50bc-4eb2-80e6-8c98d359efd1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3ef53455 2021-08-11T16:38:23.181Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8f2b7a1-50bc-4eb2-80e6-8c98d359efd1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:38:23.181Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e8f2b7a1-50bc-4eb2-80e6-8c98d359efd1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:38:24.275Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=217b9885-aec0-47ad-902f-2fc0a8f4d7c6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:38:24.275Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=217b9885-aec0-47ad-902f-2fc0a8f4d7c6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@605d3299 2021-08-11T16:38:24.275Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=217b9885-aec0-47ad-902f-2fc0a8f4d7c6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:38:24.275Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=217b9885-aec0-47ad-902f-2fc0a8f4d7c6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:38:25.570Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e69d3dad-9e73-4e01-aad7-b371358f56e7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:38:25.570Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e69d3dad-9e73-4e01-aad7-b371358f56e7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@60446878 2021-08-11T16:38:25.570Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e69d3dad-9e73-4e01-aad7-b371358f56e7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:38:25.571Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e69d3dad-9e73-4e01-aad7-b371358f56e7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:38:26.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=941cc19a-54d3-4989-9eed-cc6417f04ac2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:38:26.784Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=941cc19a-54d3-4989-9eed-cc6417f04ac2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@46489c6e 2021-08-11T16:38:26.784Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=941cc19a-54d3-4989-9eed-cc6417f04ac2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:38:26.784Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=941cc19a-54d3-4989-9eed-cc6417f04ac2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:38:27.561Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9fcf9b4f-5c72-4914-9834-2fcc955f3b74 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:38:27.561Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9fcf9b4f-5c72-4914-9834-2fcc955f3b74 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@51884f6e 2021-08-11T16:38:27.561Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9fcf9b4f-5c72-4914-9834-2fcc955f3b74 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:38:27.561Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9fcf9b4f-5c72-4914-9834-2fcc955f3b74 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:38:28.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=36296c92-539b-487a-b2f9-ae29efab205c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:38:28.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=36296c92-539b-487a-b2f9-ae29efab205c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5848763c 2021-08-11T16:38:28.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=36296c92-539b-487a-b2f9-ae29efab205c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:38:28.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=36296c92-539b-487a-b2f9-ae29efab205c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:38:31.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b425080d-1d07-40bb-8d1b-493144449bb0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:38:31.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b425080d-1d07-40bb-8d1b-493144449bb0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1b8cd2e5 2021-08-11T16:38:31.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b425080d-1d07-40bb-8d1b-493144449bb0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:38:31.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b425080d-1d07-40bb-8d1b-493144449bb0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:38:33.305Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:38:33.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:38:34.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a02872f2-88fb-4c2e-9964-485179a8dbe6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:38:34.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a02872f2-88fb-4c2e-9964-485179a8dbe6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@19a58616 2021-08-11T16:38:34.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a02872f2-88fb-4c2e-9964-485179a8dbe6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:38:34.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a02872f2-88fb-4c2e-9964-485179a8dbe6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:38:37.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e86e4ed-3557-43a1-8401-8cfc2710c20c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:38:37.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e86e4ed-3557-43a1-8401-8cfc2710c20c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@a454973 2021-08-11T16:38:37.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e86e4ed-3557-43a1-8401-8cfc2710c20c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:38:37.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e86e4ed-3557-43a1-8401-8cfc2710c20c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:38:40.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f16691f3-b93f-4e45-9327-f899faaf634c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:38:40.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f16691f3-b93f-4e45-9327-f899faaf634c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@11a72791 2021-08-11T16:38:40.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f16691f3-b93f-4e45-9327-f899faaf634c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:38:40.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f16691f3-b93f-4e45-9327-f899faaf634c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:38:43.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c117e5d-c39c-4681-961f-600b3965f19f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:38:43.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c117e5d-c39c-4681-961f-600b3965f19f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6c41c4ba 2021-08-11T16:38:43.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c117e5d-c39c-4681-961f-600b3965f19f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:38:43.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c117e5d-c39c-4681-961f-600b3965f19f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:38:46.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=15659c21-370f-45e4-8a04-0405750c4c08 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:38:46.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=15659c21-370f-45e4-8a04-0405750c4c08 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@59e17a12 2021-08-11T16:38:46.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=15659c21-370f-45e4-8a04-0405750c4c08 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:38:46.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=15659c21-370f-45e4-8a04-0405750c4c08 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:38:48.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:38:48.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:38:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:38:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:38:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:38:49.949Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cc447cc8-c6e8-4f09-b0f5-7d852a968d24 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:38:49.949Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cc447cc8-c6e8-4f09-b0f5-7d852a968d24 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@79cfbe8c 2021-08-11T16:38:49.949Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cc447cc8-c6e8-4f09-b0f5-7d852a968d24 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:38:49.949Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cc447cc8-c6e8-4f09-b0f5-7d852a968d24 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:38:52.824Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f62b3c91-75c2-4468-b1f5-ea7772cb632f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:38:52.824Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f62b3c91-75c2-4468-b1f5-ea7772cb632f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@517c958b 2021-08-11T16:38:52.824Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f62b3c91-75c2-4468-b1f5-ea7772cb632f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:38:52.824Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f62b3c91-75c2-4468-b1f5-ea7772cb632f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:38:55.892Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2be8657-f356-4b8d-b895-cf1e0cbf5ae9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:38:55.892Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2be8657-f356-4b8d-b895-cf1e0cbf5ae9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@33faff0a 2021-08-11T16:38:55.892Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2be8657-f356-4b8d-b895-cf1e0cbf5ae9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:38:55.892Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2be8657-f356-4b8d-b895-cf1e0cbf5ae9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:38:58.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56c704d6-66af-4ea3-9e49-771d83c99ea8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:38:58.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56c704d6-66af-4ea3-9e49-771d83c99ea8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6119228d 2021-08-11T16:38:58.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56c704d6-66af-4ea3-9e49-771d83c99ea8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:38:58.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=56c704d6-66af-4ea3-9e49-771d83c99ea8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:39:01.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=561525af-170e-4fc1-83ae-37bcea8e00ab PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:39:01.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=561525af-170e-4fc1-83ae-37bcea8e00ab PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@65a6dbfc 2021-08-11T16:39:01.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=561525af-170e-4fc1-83ae-37bcea8e00ab PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:39:01.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=561525af-170e-4fc1-83ae-37bcea8e00ab PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:39:03.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:39:03.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:39:04.249Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:39:04.711Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31c28d22-f122-4d78-93bd-a0991e26801c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:39:04.711Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31c28d22-f122-4d78-93bd-a0991e26801c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5043ca0e 2021-08-11T16:39:04.711Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31c28d22-f122-4d78-93bd-a0991e26801c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:39:04.711Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=31c28d22-f122-4d78-93bd-a0991e26801c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:39:07.714Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e53cc91-6a5c-44e2-b561-9dfca43c892f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:39:07.714Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e53cc91-6a5c-44e2-b561-9dfca43c892f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@34a56da3 2021-08-11T16:39:07.714Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e53cc91-6a5c-44e2-b561-9dfca43c892f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:39:07.715Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e53cc91-6a5c-44e2-b561-9dfca43c892f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:39:10.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b414a4e7-0bc4-4d5e-88b6-89610a20acd8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:39:10.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b414a4e7-0bc4-4d5e-88b6-89610a20acd8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1c803cf0 2021-08-11T16:39:10.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b414a4e7-0bc4-4d5e-88b6-89610a20acd8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:39:10.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b414a4e7-0bc4-4d5e-88b6-89610a20acd8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:39:13.637Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=238391ec-7652-4ec0-92e2-e7bcd0ff1d49 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:39:13.637Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=238391ec-7652-4ec0-92e2-e7bcd0ff1d49 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4faa6da7 2021-08-11T16:39:13.637Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=238391ec-7652-4ec0-92e2-e7bcd0ff1d49 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:39:13.637Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=238391ec-7652-4ec0-92e2-e7bcd0ff1d49 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:39:16.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5cadc9f9-6a30-4117-8b31-120d10eb45a4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:39:16.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5cadc9f9-6a30-4117-8b31-120d10eb45a4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1f7dde6 2021-08-11T16:39:16.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5cadc9f9-6a30-4117-8b31-120d10eb45a4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:39:16.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5cadc9f9-6a30-4117-8b31-120d10eb45a4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:39:18.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:39:18.319Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:39:19.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02c2fc9b-000d-4020-8bde-44bd7cceb977 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:39:19.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02c2fc9b-000d-4020-8bde-44bd7cceb977 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1ea0700e 2021-08-11T16:39:19.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02c2fc9b-000d-4020-8bde-44bd7cceb977 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:39:19.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=02c2fc9b-000d-4020-8bde-44bd7cceb977 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:39:22.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2596cc95-06e9-4077-bb66-a19552f3888f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:39:22.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2596cc95-06e9-4077-bb66-a19552f3888f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@35aa896 2021-08-11T16:39:22.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2596cc95-06e9-4077-bb66-a19552f3888f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:39:22.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2596cc95-06e9-4077-bb66-a19552f3888f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:39:25.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf37e17f-351d-4c84-bfc0-7a78ef10d865 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:39:25.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf37e17f-351d-4c84-bfc0-7a78ef10d865 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1f012a0f 2021-08-11T16:39:25.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf37e17f-351d-4c84-bfc0-7a78ef10d865 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:39:25.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cf37e17f-351d-4c84-bfc0-7a78ef10d865 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:39:28.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7623ec9a-12cd-408b-acab-d64296956c85 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:39:28.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7623ec9a-12cd-408b-acab-d64296956c85 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2026dce4 2021-08-11T16:39:28.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7623ec9a-12cd-408b-acab-d64296956c85 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:39:28.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7623ec9a-12cd-408b-acab-d64296956c85 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:39:31.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2e2bc3f-7063-4892-8ed9-cc90c4aaff8a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:39:31.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2e2bc3f-7063-4892-8ed9-cc90c4aaff8a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3d1bf8ee 2021-08-11T16:39:31.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2e2bc3f-7063-4892-8ed9-cc90c4aaff8a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:39:31.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2e2bc3f-7063-4892-8ed9-cc90c4aaff8a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:39:33.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:39:33.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:39:34.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c74455f7-b429-452b-8126-d1456562f17c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:39:34.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c74455f7-b429-452b-8126-d1456562f17c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6b53f9c3 2021-08-11T16:39:34.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c74455f7-b429-452b-8126-d1456562f17c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:39:34.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c74455f7-b429-452b-8126-d1456562f17c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:39:37.755Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82ada1cd-5425-49e2-8ef6-07307197c0b6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:39:37.755Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82ada1cd-5425-49e2-8ef6-07307197c0b6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7d14f7f4 2021-08-11T16:39:37.755Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82ada1cd-5425-49e2-8ef6-07307197c0b6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:39:37.755Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82ada1cd-5425-49e2-8ef6-07307197c0b6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:39:40.850Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb2c5595-873f-4c1f-964e-eb799a4ad5de PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:39:40.850Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb2c5595-873f-4c1f-964e-eb799a4ad5de PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6bc3c54e 2021-08-11T16:39:40.851Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb2c5595-873f-4c1f-964e-eb799a4ad5de PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:39:40.851Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb2c5595-873f-4c1f-964e-eb799a4ad5de PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:39:43.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b13addc-c4cb-4533-ac73-ed5e0b0d5631 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:39:43.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b13addc-c4cb-4533-ac73-ed5e0b0d5631 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@9fd9c50 2021-08-11T16:39:43.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b13addc-c4cb-4533-ac73-ed5e0b0d5631 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:39:43.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b13addc-c4cb-4533-ac73-ed5e0b0d5631 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:39:46.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ada8a0b6-66ba-4061-a854-2f00ecbfebad PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:39:46.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ada8a0b6-66ba-4061-a854-2f00ecbfebad PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7d94f14d 2021-08-11T16:39:46.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ada8a0b6-66ba-4061-a854-2f00ecbfebad PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:39:46.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ada8a0b6-66ba-4061-a854-2f00ecbfebad PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:39:48.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:39:48.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:39:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:39:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:39:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:39:49.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ddefcf25-9c58-450a-9357-8835258c3d48 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:39:49.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ddefcf25-9c58-450a-9357-8835258c3d48 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@27e5025c 2021-08-11T16:39:49.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ddefcf25-9c58-450a-9357-8835258c3d48 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:39:49.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ddefcf25-9c58-450a-9357-8835258c3d48 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:39:53.170Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1cf8e16-a0d5-4020-b4ef-9640932d7782 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:39:53.170Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1cf8e16-a0d5-4020-b4ef-9640932d7782 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@52794a5b 2021-08-11T16:39:53.171Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1cf8e16-a0d5-4020-b4ef-9640932d7782 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:39:53.171Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1cf8e16-a0d5-4020-b4ef-9640932d7782 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:39:56.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=911757fe-557d-40ae-9a80-572e4749d42a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:39:56.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=911757fe-557d-40ae-9a80-572e4749d42a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@37035358 2021-08-11T16:39:56.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=911757fe-557d-40ae-9a80-572e4749d42a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:39:56.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=911757fe-557d-40ae-9a80-572e4749d42a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:39:59.367Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ae99722-a25d-4184-bd8e-cfb40176dc94 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:39:59.367Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ae99722-a25d-4184-bd8e-cfb40176dc94 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6dc7c26c 2021-08-11T16:39:59.367Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ae99722-a25d-4184-bd8e-cfb40176dc94 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:39:59.367Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ae99722-a25d-4184-bd8e-cfb40176dc94 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:40:01.926Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f55a35f-d65f-493a-a714-6797039893c3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:40:01.926Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f55a35f-d65f-493a-a714-6797039893c3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@62bfda6a 2021-08-11T16:40:01.926Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f55a35f-d65f-493a-a714-6797039893c3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:40:01.926Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f55a35f-d65f-493a-a714-6797039893c3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:40:03.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:40:03.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:40:04.256Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:40:04.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=27d42d00-1357-486f-8954-582525158519 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:40:04.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=27d42d00-1357-486f-8954-582525158519 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7a8c7ff4 2021-08-11T16:40:04.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=27d42d00-1357-486f-8954-582525158519 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:40:04.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=27d42d00-1357-486f-8954-582525158519 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:40:07.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8b57d809-fd96-420e-aa65-4dc8c4ba4298 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:40:07.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8b57d809-fd96-420e-aa65-4dc8c4ba4298 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@57b86a6e 2021-08-11T16:40:07.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8b57d809-fd96-420e-aa65-4dc8c4ba4298 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:40:07.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8b57d809-fd96-420e-aa65-4dc8c4ba4298 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:40:10.627Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1115d83e-d6d8-4312-bb03-c33ca7247d10 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:40:10.627Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1115d83e-d6d8-4312-bb03-c33ca7247d10 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@43f1657 2021-08-11T16:40:10.628Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1115d83e-d6d8-4312-bb03-c33ca7247d10 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:40:10.628Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1115d83e-d6d8-4312-bb03-c33ca7247d10 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:40:13.627Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=30fad7c6-41a2-4015-af6c-11bf1e392a40 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:40:13.628Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=30fad7c6-41a2-4015-af6c-11bf1e392a40 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@184ff030 2021-08-11T16:40:13.628Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=30fad7c6-41a2-4015-af6c-11bf1e392a40 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:40:13.628Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=30fad7c6-41a2-4015-af6c-11bf1e392a40 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:40:16.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0ab2153-4e5a-449d-ad1f-4b125dba2dbc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:40:16.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0ab2153-4e5a-449d-ad1f-4b125dba2dbc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1a55f12 2021-08-11T16:40:16.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0ab2153-4e5a-449d-ad1f-4b125dba2dbc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:40:16.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0ab2153-4e5a-449d-ad1f-4b125dba2dbc PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:40:18.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:40:18.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:40:19.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=84374c07-ec84-4f5b-9bbf-2fc94ac4bb35 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:40:19.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=84374c07-ec84-4f5b-9bbf-2fc94ac4bb35 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@15dabe52 2021-08-11T16:40:19.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=84374c07-ec84-4f5b-9bbf-2fc94ac4bb35 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:40:19.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=84374c07-ec84-4f5b-9bbf-2fc94ac4bb35 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:40:22.687Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=183ef5a6-fe12-447a-a749-3c88c2f6f13f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:40:22.687Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=183ef5a6-fe12-447a-a749-3c88c2f6f13f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@69ab9f10 2021-08-11T16:40:22.687Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=183ef5a6-fe12-447a-a749-3c88c2f6f13f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:40:22.687Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=183ef5a6-fe12-447a-a749-3c88c2f6f13f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:40:25.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f0b8462-bc42-488b-b60f-97f44ead001f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:40:25.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f0b8462-bc42-488b-b60f-97f44ead001f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@e023c65 2021-08-11T16:40:25.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f0b8462-bc42-488b-b60f-97f44ead001f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:40:25.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f0b8462-bc42-488b-b60f-97f44ead001f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:40:28.852Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47d4d99b-7d5b-408d-8e5b-b322e5efe72c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:40:28.852Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47d4d99b-7d5b-408d-8e5b-b322e5efe72c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@25f4bf78 2021-08-11T16:40:28.852Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47d4d99b-7d5b-408d-8e5b-b322e5efe72c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:40:28.852Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47d4d99b-7d5b-408d-8e5b-b322e5efe72c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:40:31.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7db0038b-09d6-4267-86dc-46c1cc3d8efa PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:40:31.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7db0038b-09d6-4267-86dc-46c1cc3d8efa PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@651cbdba 2021-08-11T16:40:31.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7db0038b-09d6-4267-86dc-46c1cc3d8efa PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:40:31.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7db0038b-09d6-4267-86dc-46c1cc3d8efa PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:40:33.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:40:33.318Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:40:34.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf0a1260-8fa8-4050-848c-b3fa7bb0e62e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:40:34.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf0a1260-8fa8-4050-848c-b3fa7bb0e62e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@50faed16 2021-08-11T16:40:34.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf0a1260-8fa8-4050-848c-b3fa7bb0e62e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:40:34.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf0a1260-8fa8-4050-848c-b3fa7bb0e62e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:40:37.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=92cdf434-82a3-4024-b70b-a82a029a8a85 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:40:37.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=92cdf434-82a3-4024-b70b-a82a029a8a85 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4f841278 2021-08-11T16:40:37.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=92cdf434-82a3-4024-b70b-a82a029a8a85 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:40:37.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=92cdf434-82a3-4024-b70b-a82a029a8a85 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:40:40.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=28c066f9-15c6-4dd3-a18f-f7751192cfd9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:40:40.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=28c066f9-15c6-4dd3-a18f-f7751192cfd9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4f0b7b8f 2021-08-11T16:40:40.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=28c066f9-15c6-4dd3-a18f-f7751192cfd9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:40:40.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=28c066f9-15c6-4dd3-a18f-f7751192cfd9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:40:43.617Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db23f310-52c5-498f-9458-9cdef326639a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:40:43.617Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db23f310-52c5-498f-9458-9cdef326639a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4ffaeeb9 2021-08-11T16:40:43.617Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db23f310-52c5-498f-9458-9cdef326639a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:40:43.617Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db23f310-52c5-498f-9458-9cdef326639a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:40:46.633Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79b8a289-4e5e-411f-b369-f7a77efa460c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:40:46.633Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79b8a289-4e5e-411f-b369-f7a77efa460c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@48ef280d 2021-08-11T16:40:46.633Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79b8a289-4e5e-411f-b369-f7a77efa460c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:40:46.633Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79b8a289-4e5e-411f-b369-f7a77efa460c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:40:48.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:40:48.317Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:40:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:40:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:40:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:40:50.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ceefd93-425e-4ae1-9c5c-d0014053d29d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:40:50.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ceefd93-425e-4ae1-9c5c-d0014053d29d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2717f4ba 2021-08-11T16:40:50.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ceefd93-425e-4ae1-9c5c-d0014053d29d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:40:50.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ceefd93-425e-4ae1-9c5c-d0014053d29d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:40:53.554Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=61a5ea87-fbfe-47a0-8a4c-5b81f9578345 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:40:53.554Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=61a5ea87-fbfe-47a0-8a4c-5b81f9578345 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@33d424d4 2021-08-11T16:40:53.555Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=61a5ea87-fbfe-47a0-8a4c-5b81f9578345 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:40:53.555Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=61a5ea87-fbfe-47a0-8a4c-5b81f9578345 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:40:56.968Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea1da438-ac11-4f5a-980b-54923a92b43a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:40:56.968Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea1da438-ac11-4f5a-980b-54923a92b43a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2bc38b31 2021-08-11T16:40:56.968Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea1da438-ac11-4f5a-980b-54923a92b43a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:40:56.968Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea1da438-ac11-4f5a-980b-54923a92b43a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:40:59.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=af9bdf7a-0712-4b7b-bb07-8bd792fd4a72 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:40:59.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=af9bdf7a-0712-4b7b-bb07-8bd792fd4a72 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1add7104 2021-08-11T16:40:59.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=af9bdf7a-0712-4b7b-bb07-8bd792fd4a72 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:40:59.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=af9bdf7a-0712-4b7b-bb07-8bd792fd4a72 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:41:02.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44f863d5-a1dc-4dbe-9f09-7bae1ee3f97c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:41:02.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44f863d5-a1dc-4dbe-9f09-7bae1ee3f97c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2f7e4c1 2021-08-11T16:41:02.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44f863d5-a1dc-4dbe-9f09-7bae1ee3f97c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:41:02.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=44f863d5-a1dc-4dbe-9f09-7bae1ee3f97c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:41:03.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:41:03.323Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:41:04.260Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:41:05.365Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b208359-4812-4489-bddb-9f877a1e8e75 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:41:05.365Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b208359-4812-4489-bddb-9f877a1e8e75 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@109782cf 2021-08-11T16:41:05.365Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b208359-4812-4489-bddb-9f877a1e8e75 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:41:05.365Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b208359-4812-4489-bddb-9f877a1e8e75 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:41:07.799Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01c971f1-eb55-4565-8363-77d7e64f3f90 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:41:07.799Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01c971f1-eb55-4565-8363-77d7e64f3f90 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@27ba19ef 2021-08-11T16:41:07.799Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01c971f1-eb55-4565-8363-77d7e64f3f90 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:41:07.799Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01c971f1-eb55-4565-8363-77d7e64f3f90 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:41:10.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a2c034bb-11a4-4860-976b-5f836d219fa6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:41:10.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a2c034bb-11a4-4860-976b-5f836d219fa6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@561e094d 2021-08-11T16:41:10.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a2c034bb-11a4-4860-976b-5f836d219fa6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:41:10.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a2c034bb-11a4-4860-976b-5f836d219fa6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:41:13.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a459a8b2-1f65-4ac8-9000-1f60b1763b7f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:41:13.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a459a8b2-1f65-4ac8-9000-1f60b1763b7f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6bf23ea3 2021-08-11T16:41:13.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a459a8b2-1f65-4ac8-9000-1f60b1763b7f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:41:13.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a459a8b2-1f65-4ac8-9000-1f60b1763b7f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:41:16.633Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=23e2f7ae-f487-41e7-8d1e-04e6798e4ebd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:41:16.633Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=23e2f7ae-f487-41e7-8d1e-04e6798e4ebd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@47df48e8 2021-08-11T16:41:16.633Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=23e2f7ae-f487-41e7-8d1e-04e6798e4ebd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:41:16.634Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=23e2f7ae-f487-41e7-8d1e-04e6798e4ebd PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:41:18.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:41:18.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:41:19.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b67625e0-1d6b-4606-b531-d3d0f728e0ca PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:41:19.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b67625e0-1d6b-4606-b531-d3d0f728e0ca PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@30dd449d 2021-08-11T16:41:19.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b67625e0-1d6b-4606-b531-d3d0f728e0ca PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:41:19.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b67625e0-1d6b-4606-b531-d3d0f728e0ca PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:41:22.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=60e0032a-9ed9-4305-9aa2-5a3aebde51f8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:41:22.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=60e0032a-9ed9-4305-9aa2-5a3aebde51f8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@162f67aa 2021-08-11T16:41:22.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=60e0032a-9ed9-4305-9aa2-5a3aebde51f8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:41:22.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=60e0032a-9ed9-4305-9aa2-5a3aebde51f8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:41:25.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9dd6d11f-b7b6-412e-b2a9-56fda5ef47c9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:41:25.652Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9dd6d11f-b7b6-412e-b2a9-56fda5ef47c9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6cd244ea 2021-08-11T16:41:25.652Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9dd6d11f-b7b6-412e-b2a9-56fda5ef47c9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:41:25.652Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9dd6d11f-b7b6-412e-b2a9-56fda5ef47c9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:41:28.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f918713f-e294-46a2-85b6-d4f38fa54643 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:41:28.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f918713f-e294-46a2-85b6-d4f38fa54643 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5177c990 2021-08-11T16:41:28.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f918713f-e294-46a2-85b6-d4f38fa54643 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:41:28.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f918713f-e294-46a2-85b6-d4f38fa54643 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:41:31.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c685890-fc4a-4ca7-bdb3-59bbcabd2da6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:41:31.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c685890-fc4a-4ca7-bdb3-59bbcabd2da6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@bd4868f 2021-08-11T16:41:31.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c685890-fc4a-4ca7-bdb3-59bbcabd2da6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:41:31.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c685890-fc4a-4ca7-bdb3-59bbcabd2da6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:41:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:41:33.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:41:34.746Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=306bf596-9061-4f18-8d9c-0dbfeb9c1a6c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:41:34.746Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=306bf596-9061-4f18-8d9c-0dbfeb9c1a6c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@66abb268 2021-08-11T16:41:34.746Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=306bf596-9061-4f18-8d9c-0dbfeb9c1a6c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:41:34.746Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=306bf596-9061-4f18-8d9c-0dbfeb9c1a6c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:41:37.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73a8c931-1427-45b7-b48b-7c48091cf0c5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:41:37.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73a8c931-1427-45b7-b48b-7c48091cf0c5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@18204f43 2021-08-11T16:41:37.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73a8c931-1427-45b7-b48b-7c48091cf0c5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:41:37.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73a8c931-1427-45b7-b48b-7c48091cf0c5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:41:40.948Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6bcb9514-a211-4780-8478-a9c537be5760 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:41:40.948Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6bcb9514-a211-4780-8478-a9c537be5760 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@77742e9e 2021-08-11T16:41:40.948Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6bcb9514-a211-4780-8478-a9c537be5760 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:41:40.948Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6bcb9514-a211-4780-8478-a9c537be5760 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:41:43.635Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b67a51b0-41ad-47bb-b244-b472360f57b2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:41:43.635Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b67a51b0-41ad-47bb-b244-b472360f57b2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@58bb2e01 2021-08-11T16:41:43.635Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b67a51b0-41ad-47bb-b244-b472360f57b2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:41:43.635Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b67a51b0-41ad-47bb-b244-b472360f57b2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:41:46.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b4b0f69-51aa-4010-ab44-cc53cc08cd98 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:41:46.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b4b0f69-51aa-4010-ab44-cc53cc08cd98 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6338ff8c 2021-08-11T16:41:46.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b4b0f69-51aa-4010-ab44-cc53cc08cd98 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:41:46.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b4b0f69-51aa-4010-ab44-cc53cc08cd98 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:41:48.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:41:48.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:41:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:41:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:41:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:41:49.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=20186f7a-d8ad-4fb2-9b06-309421380f96 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:41:49.956Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=20186f7a-d8ad-4fb2-9b06-309421380f96 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@79f8feff 2021-08-11T16:41:49.956Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=20186f7a-d8ad-4fb2-9b06-309421380f96 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:41:49.956Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=20186f7a-d8ad-4fb2-9b06-309421380f96 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:41:53.367Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=84709bea-1af0-4aa3-8d25-531584461b17 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:41:53.367Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=84709bea-1af0-4aa3-8d25-531584461b17 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2fae3f80 2021-08-11T16:41:53.367Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=84709bea-1af0-4aa3-8d25-531584461b17 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:41:53.367Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=84709bea-1af0-4aa3-8d25-531584461b17 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:41:56.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=65e720d7-3351-47d8-a255-b5cffc2e38d0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:41:56.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=65e720d7-3351-47d8-a255-b5cffc2e38d0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2e64c752 2021-08-11T16:41:56.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=65e720d7-3351-47d8-a255-b5cffc2e38d0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:41:56.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=65e720d7-3351-47d8-a255-b5cffc2e38d0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:41:59.270Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1bf7bfc6-623b-41ec-9c39-56b3e458922c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:41:59.270Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1bf7bfc6-623b-41ec-9c39-56b3e458922c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@a04898d 2021-08-11T16:41:59.270Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1bf7bfc6-623b-41ec-9c39-56b3e458922c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:41:59.270Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1bf7bfc6-623b-41ec-9c39-56b3e458922c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:42:01.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=779a8239-bde9-4d94-9ec6-a04d33b00532 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:42:01.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=779a8239-bde9-4d94-9ec6-a04d33b00532 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@34358134 2021-08-11T16:42:01.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=779a8239-bde9-4d94-9ec6-a04d33b00532 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:42:01.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=779a8239-bde9-4d94-9ec6-a04d33b00532 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:42:03.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:42:03.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:42:04.273Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:42:04.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa702d67-fc40-4e19-ad01-625fa3c3a058 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:42:04.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa702d67-fc40-4e19-ad01-625fa3c3a058 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6ab9ea0d 2021-08-11T16:42:04.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa702d67-fc40-4e19-ad01-625fa3c3a058 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:42:04.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa702d67-fc40-4e19-ad01-625fa3c3a058 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:42:07.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=437cae6a-90a7-4aba-a125-915dbcdf9cf8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:42:07.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=437cae6a-90a7-4aba-a125-915dbcdf9cf8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@626ffe5f 2021-08-11T16:42:07.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=437cae6a-90a7-4aba-a125-915dbcdf9cf8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:42:07.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=437cae6a-90a7-4aba-a125-915dbcdf9cf8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:42:10.613Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea87ebbb-3afe-4a99-bd16-763d2396e875 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:42:10.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea87ebbb-3afe-4a99-bd16-763d2396e875 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4c095b7e 2021-08-11T16:42:10.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea87ebbb-3afe-4a99-bd16-763d2396e875 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:42:10.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea87ebbb-3afe-4a99-bd16-763d2396e875 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:42:13.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5d4bf8a4-4af3-414e-b2fe-a8ee592c437e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:42:13.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5d4bf8a4-4af3-414e-b2fe-a8ee592c437e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@54ad4fea 2021-08-11T16:42:13.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5d4bf8a4-4af3-414e-b2fe-a8ee592c437e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:42:13.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5d4bf8a4-4af3-414e-b2fe-a8ee592c437e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:42:16.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f24c1040-6c16-4e26-b671-db3f61a75f6b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:42:16.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f24c1040-6c16-4e26-b671-db3f61a75f6b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@71613ef0 2021-08-11T16:42:16.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f24c1040-6c16-4e26-b671-db3f61a75f6b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:42:16.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f24c1040-6c16-4e26-b671-db3f61a75f6b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:42:18.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:42:18.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:42:19.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f65527ec-2af0-4ba2-bd7b-457953a880d9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:42:19.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f65527ec-2af0-4ba2-bd7b-457953a880d9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6ce1bb85 2021-08-11T16:42:19.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f65527ec-2af0-4ba2-bd7b-457953a880d9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:42:19.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f65527ec-2af0-4ba2-bd7b-457953a880d9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:42:22.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd01e3bb-7866-45eb-8bd4-3716b0049897 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:42:22.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd01e3bb-7866-45eb-8bd4-3716b0049897 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6a35dfcc 2021-08-11T16:42:22.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd01e3bb-7866-45eb-8bd4-3716b0049897 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:42:22.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd01e3bb-7866-45eb-8bd4-3716b0049897 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:42:25.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=687a02c6-056a-4e42-b047-43f3de349a71 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:42:25.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=687a02c6-056a-4e42-b047-43f3de349a71 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@54590f7f 2021-08-11T16:42:25.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=687a02c6-056a-4e42-b047-43f3de349a71 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:42:25.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=687a02c6-056a-4e42-b047-43f3de349a71 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:42:28.949Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=afe18387-f875-4d8f-b519-01aaa8642012 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:42:28.949Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=afe18387-f875-4d8f-b519-01aaa8642012 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2c372dba 2021-08-11T16:42:28.949Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=afe18387-f875-4d8f-b519-01aaa8642012 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:42:28.949Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=afe18387-f875-4d8f-b519-01aaa8642012 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:42:31.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5a6af54-9e8d-4188-9596-0be41df12da0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:42:31.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5a6af54-9e8d-4188-9596-0be41df12da0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4deb962a 2021-08-11T16:42:31.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5a6af54-9e8d-4188-9596-0be41df12da0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:42:31.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5a6af54-9e8d-4188-9596-0be41df12da0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:42:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:42:33.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:42:34.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70534066-d2a3-4274-8c05-f8cfb31cc3a8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:42:34.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70534066-d2a3-4274-8c05-f8cfb31cc3a8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@964ebe4 2021-08-11T16:42:34.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70534066-d2a3-4274-8c05-f8cfb31cc3a8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:42:34.632Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70534066-d2a3-4274-8c05-f8cfb31cc3a8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:42:37.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8b2396c0-e7ec-4180-8cb3-12e3a90a9031 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:42:37.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8b2396c0-e7ec-4180-8cb3-12e3a90a9031 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@15da3f40 2021-08-11T16:42:37.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8b2396c0-e7ec-4180-8cb3-12e3a90a9031 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:42:37.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8b2396c0-e7ec-4180-8cb3-12e3a90a9031 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:42:40.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a67cf7a-9a7b-4244-a1a1-462de479b6bd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:42:40.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a67cf7a-9a7b-4244-a1a1-462de479b6bd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@958abc 2021-08-11T16:42:40.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a67cf7a-9a7b-4244-a1a1-462de479b6bd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:42:40.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a67cf7a-9a7b-4244-a1a1-462de479b6bd PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:42:43.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e713327-11c8-4294-a542-a0a176b9baec PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:42:43.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e713327-11c8-4294-a542-a0a176b9baec PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@11de0642 2021-08-11T16:42:43.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e713327-11c8-4294-a542-a0a176b9baec PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:42:43.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6e713327-11c8-4294-a542-a0a176b9baec PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:42:46.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9278db9c-549f-428b-8f78-12f10ff4c2c5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:42:46.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9278db9c-549f-428b-8f78-12f10ff4c2c5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3fcbe456 2021-08-11T16:42:46.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9278db9c-549f-428b-8f78-12f10ff4c2c5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:42:46.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9278db9c-549f-428b-8f78-12f10ff4c2c5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:42:48.309Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:42:48.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:42:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:42:49.292Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:42:49.292Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:42:50.347Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1e0fc2c-64fc-4a39-9c0f-d7bfe7359411 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:42:50.347Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1e0fc2c-64fc-4a39-9c0f-d7bfe7359411 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7bafecd0 2021-08-11T16:42:50.348Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1e0fc2c-64fc-4a39-9c0f-d7bfe7359411 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:42:50.348Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1e0fc2c-64fc-4a39-9c0f-d7bfe7359411 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:42:53.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e196540f-9329-4aa5-b91b-e21d64d67ddf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:42:53.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e196540f-9329-4aa5-b91b-e21d64d67ddf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@57435973 2021-08-11T16:42:53.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e196540f-9329-4aa5-b91b-e21d64d67ddf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:42:53.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e196540f-9329-4aa5-b91b-e21d64d67ddf PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:42:57.167Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=803de747-b9f0-43d8-8371-5361a97ca8c5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:42:57.167Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=803de747-b9f0-43d8-8371-5361a97ca8c5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@36ceb9cf 2021-08-11T16:42:57.167Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=803de747-b9f0-43d8-8371-5361a97ca8c5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:42:57.168Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=803de747-b9f0-43d8-8371-5361a97ca8c5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:42:59.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=beb00cf9-baee-4570-8b14-d919266c4b20 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:42:59.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=beb00cf9-baee-4570-8b14-d919266c4b20 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@56c71214 2021-08-11T16:42:59.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=beb00cf9-baee-4570-8b14-d919266c4b20 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:42:59.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=beb00cf9-baee-4570-8b14-d919266c4b20 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:43:02.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=27ad00d7-61b9-4cc8-a89f-69086bbc9188 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:43:02.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=27ad00d7-61b9-4cc8-a89f-69086bbc9188 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@528abd70 2021-08-11T16:43:02.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=27ad00d7-61b9-4cc8-a89f-69086bbc9188 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:43:02.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=27ad00d7-61b9-4cc8-a89f-69086bbc9188 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:43:03.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:43:03.311Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:43:04.271Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:43:05.382Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=190a80c7-1a09-41df-8121-104ea4e3c919 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:43:05.382Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=190a80c7-1a09-41df-8121-104ea4e3c919 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1fcfee93 2021-08-11T16:43:05.382Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=190a80c7-1a09-41df-8121-104ea4e3c919 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:43:05.382Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=190a80c7-1a09-41df-8121-104ea4e3c919 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:43:07.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24c9d2fe-362b-4a22-a741-62858f715331 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:43:07.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24c9d2fe-362b-4a22-a741-62858f715331 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5c0c2c64 2021-08-11T16:43:07.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24c9d2fe-362b-4a22-a741-62858f715331 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:43:07.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24c9d2fe-362b-4a22-a741-62858f715331 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:43:10.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc026206-33dd-440c-a4e6-ffb2c3363180 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:43:10.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc026206-33dd-440c-a4e6-ffb2c3363180 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3235d243 2021-08-11T16:43:10.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc026206-33dd-440c-a4e6-ffb2c3363180 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:43:10.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc026206-33dd-440c-a4e6-ffb2c3363180 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:43:13.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21a56954-434a-462d-bdca-93ead336b756 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:43:13.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21a56954-434a-462d-bdca-93ead336b756 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@35d39848 2021-08-11T16:43:13.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21a56954-434a-462d-bdca-93ead336b756 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:43:13.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21a56954-434a-462d-bdca-93ead336b756 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:43:16.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0daed246-84d8-4c7d-8a72-1ac471a42289 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:43:16.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0daed246-84d8-4c7d-8a72-1ac471a42289 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@55085317 2021-08-11T16:43:16.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0daed246-84d8-4c7d-8a72-1ac471a42289 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:43:16.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0daed246-84d8-4c7d-8a72-1ac471a42289 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:43:18.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:43:18.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:43:19.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe50a7ec-652a-4908-b0f2-fa3f92363bb4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:43:19.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe50a7ec-652a-4908-b0f2-fa3f92363bb4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@68da3ade 2021-08-11T16:43:19.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe50a7ec-652a-4908-b0f2-fa3f92363bb4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:43:19.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe50a7ec-652a-4908-b0f2-fa3f92363bb4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:43:23.362Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc47c37f-b084-4268-a626-89695f8830c8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:43:23.362Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc47c37f-b084-4268-a626-89695f8830c8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@74614a4c 2021-08-11T16:43:23.362Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc47c37f-b084-4268-a626-89695f8830c8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:43:23.362Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc47c37f-b084-4268-a626-89695f8830c8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:43:25.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=832fb259-f1d8-4ca8-957c-341bc430a1f2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:43:25.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=832fb259-f1d8-4ca8-957c-341bc430a1f2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5dd99d50 2021-08-11T16:43:25.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=832fb259-f1d8-4ca8-957c-341bc430a1f2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:43:25.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=832fb259-f1d8-4ca8-957c-341bc430a1f2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:43:28.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ea0e45b-b703-42c5-8dd4-040e8daf9fc6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:43:28.869Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ea0e45b-b703-42c5-8dd4-040e8daf9fc6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4a5eed17 2021-08-11T16:43:28.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ea0e45b-b703-42c5-8dd4-040e8daf9fc6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:43:28.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ea0e45b-b703-42c5-8dd4-040e8daf9fc6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:43:31.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e600d9e8-03b2-4a38-9054-76998e5e9681 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:43:31.632Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e600d9e8-03b2-4a38-9054-76998e5e9681 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@736b2248 2021-08-11T16:43:31.632Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e600d9e8-03b2-4a38-9054-76998e5e9681 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:43:31.632Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e600d9e8-03b2-4a38-9054-76998e5e9681 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:43:33.305Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:43:33.311Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:43:34.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bbf32f5b-93fe-4883-963d-03b1673a643a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:43:34.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bbf32f5b-93fe-4883-963d-03b1673a643a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@da63543 2021-08-11T16:43:34.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bbf32f5b-93fe-4883-963d-03b1673a643a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:43:34.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bbf32f5b-93fe-4883-963d-03b1673a643a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:43:37.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ef86768-4e13-41f3-8e37-058c3d4fd7a8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:43:37.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ef86768-4e13-41f3-8e37-058c3d4fd7a8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1c040aba 2021-08-11T16:43:37.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ef86768-4e13-41f3-8e37-058c3d4fd7a8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:43:37.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ef86768-4e13-41f3-8e37-058c3d4fd7a8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:43:40.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f5f6d3c7-86de-472d-9ccd-42f01a98e334 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:43:40.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f5f6d3c7-86de-472d-9ccd-42f01a98e334 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@218f6a78 2021-08-11T16:43:40.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f5f6d3c7-86de-472d-9ccd-42f01a98e334 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:43:40.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f5f6d3c7-86de-472d-9ccd-42f01a98e334 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:43:43.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2fa2ac5-1c60-4df4-a78a-227bb298ffea PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:43:43.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2fa2ac5-1c60-4df4-a78a-227bb298ffea PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@bdc6563 2021-08-11T16:43:43.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2fa2ac5-1c60-4df4-a78a-227bb298ffea PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:43:43.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2fa2ac5-1c60-4df4-a78a-227bb298ffea PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:43:46.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f54e3d64-59ab-4bf5-9d78-b2cedb3befdc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:43:46.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f54e3d64-59ab-4bf5-9d78-b2cedb3befdc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@376c5576 2021-08-11T16:43:46.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f54e3d64-59ab-4bf5-9d78-b2cedb3befdc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:43:46.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f54e3d64-59ab-4bf5-9d78-b2cedb3befdc PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:43:48.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:43:48.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:43:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:43:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:43:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:43:50.158Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=520cb1e9-a52a-4a4d-88fd-cb9f0d648e42 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:43:50.158Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=520cb1e9-a52a-4a4d-88fd-cb9f0d648e42 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@b01d8fb 2021-08-11T16:43:50.158Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=520cb1e9-a52a-4a4d-88fd-cb9f0d648e42 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:43:50.159Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=520cb1e9-a52a-4a4d-88fd-cb9f0d648e42 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:43:53.263Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce9a6214-8925-4162-8e99-f8233c7241e0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:43:53.263Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce9a6214-8925-4162-8e99-f8233c7241e0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5107af49 2021-08-11T16:43:53.263Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce9a6214-8925-4162-8e99-f8233c7241e0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:43:53.264Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce9a6214-8925-4162-8e99-f8233c7241e0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:43:56.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e486c427-b592-468a-ab2a-5c98c057bdff PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:43:56.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e486c427-b592-468a-ab2a-5c98c057bdff PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@39983f77 2021-08-11T16:43:56.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e486c427-b592-468a-ab2a-5c98c057bdff PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:43:56.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e486c427-b592-468a-ab2a-5c98c057bdff PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:43:59.279Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9f7f7a6-3db0-49a4-a679-11f7f76e3728 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:43:59.279Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9f7f7a6-3db0-49a4-a679-11f7f76e3728 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6c0a0ad7 2021-08-11T16:43:59.279Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9f7f7a6-3db0-49a4-a679-11f7f76e3728 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:43:59.279Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9f7f7a6-3db0-49a4-a679-11f7f76e3728 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:44:01.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=98661ff5-fed7-48c7-9011-68096d8b42b2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:44:01.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=98661ff5-fed7-48c7-9011-68096d8b42b2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@69c9d1ad 2021-08-11T16:44:01.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=98661ff5-fed7-48c7-9011-68096d8b42b2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:44:01.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=98661ff5-fed7-48c7-9011-68096d8b42b2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:44:03.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:44:03.316Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:44:04.253Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:44:04.696Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe499175-bcb8-4c0c-ab74-3afe858328a2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:44:04.697Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe499175-bcb8-4c0c-ab74-3afe858328a2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@558563a 2021-08-11T16:44:04.697Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe499175-bcb8-4c0c-ab74-3afe858328a2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:44:04.697Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe499175-bcb8-4c0c-ab74-3afe858328a2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:44:07.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7ec07728-36d3-41ea-9133-b8bfcd7c370b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:44:07.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7ec07728-36d3-41ea-9133-b8bfcd7c370b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7d43b261 2021-08-11T16:44:07.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7ec07728-36d3-41ea-9133-b8bfcd7c370b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:44:07.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7ec07728-36d3-41ea-9133-b8bfcd7c370b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:44:10.713Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72a9d0d3-6cf0-47e9-9395-2fef76e4c1e4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:44:10.713Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72a9d0d3-6cf0-47e9-9395-2fef76e4c1e4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@41a353ac 2021-08-11T16:44:10.713Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72a9d0d3-6cf0-47e9-9395-2fef76e4c1e4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:44:10.713Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72a9d0d3-6cf0-47e9-9395-2fef76e4c1e4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:44:13.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=817bff37-459a-4729-9a02-148d01460051 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:44:13.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=817bff37-459a-4729-9a02-148d01460051 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6640b68c 2021-08-11T16:44:13.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=817bff37-459a-4729-9a02-148d01460051 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:44:13.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=817bff37-459a-4729-9a02-148d01460051 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:44:16.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85d91448-69d7-47f2-8907-17dfb6ba87d8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:44:16.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85d91448-69d7-47f2-8907-17dfb6ba87d8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@216ed5f8 2021-08-11T16:44:16.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85d91448-69d7-47f2-8907-17dfb6ba87d8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:44:16.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85d91448-69d7-47f2-8907-17dfb6ba87d8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:44:18.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:44:18.316Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:44:19.701Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0b05035-4538-449a-8b5c-72d8443d7b5e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:44:19.701Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0b05035-4538-449a-8b5c-72d8443d7b5e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@41deab0e 2021-08-11T16:44:19.701Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0b05035-4538-449a-8b5c-72d8443d7b5e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:44:19.701Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0b05035-4538-449a-8b5c-72d8443d7b5e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:44:22.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04ab2bc7-1a33-4780-bb38-f7033b29cd4b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:44:22.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04ab2bc7-1a33-4780-bb38-f7033b29cd4b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1a3bbeaa 2021-08-11T16:44:22.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04ab2bc7-1a33-4780-bb38-f7033b29cd4b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:44:22.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=04ab2bc7-1a33-4780-bb38-f7033b29cd4b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:44:25.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d926efef-0fd1-4593-b69d-b24daf1e0a6e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:44:25.690Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d926efef-0fd1-4593-b69d-b24daf1e0a6e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@79ac9fb7 2021-08-11T16:44:25.690Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d926efef-0fd1-4593-b69d-b24daf1e0a6e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:44:25.690Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d926efef-0fd1-4593-b69d-b24daf1e0a6e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:44:28.965Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc0b23c9-d5da-446e-8b47-15ef9f1e4d4d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:44:28.965Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc0b23c9-d5da-446e-8b47-15ef9f1e4d4d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2eba446f 2021-08-11T16:44:28.965Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc0b23c9-d5da-446e-8b47-15ef9f1e4d4d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:44:28.965Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc0b23c9-d5da-446e-8b47-15ef9f1e4d4d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:44:31.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8c6c33db-e43f-46ff-9dd6-e136dc8c1fc0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:44:31.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8c6c33db-e43f-46ff-9dd6-e136dc8c1fc0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1a080c6d 2021-08-11T16:44:31.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8c6c33db-e43f-46ff-9dd6-e136dc8c1fc0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:44:31.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8c6c33db-e43f-46ff-9dd6-e136dc8c1fc0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:44:33.309Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:44:33.319Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:44:34.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=adbdc600-6f78-4478-9ff0-6dd138a7032a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:44:34.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=adbdc600-6f78-4478-9ff0-6dd138a7032a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5d013f68 2021-08-11T16:44:34.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=adbdc600-6f78-4478-9ff0-6dd138a7032a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:44:34.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=adbdc600-6f78-4478-9ff0-6dd138a7032a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:44:37.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab243b16-72c1-4ccb-9fc8-aad27f58d709 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:44:37.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab243b16-72c1-4ccb-9fc8-aad27f58d709 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@404d4258 2021-08-11T16:44:37.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab243b16-72c1-4ccb-9fc8-aad27f58d709 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:44:37.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab243b16-72c1-4ccb-9fc8-aad27f58d709 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:44:40.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba497468-3af4-4831-bb31-b8c7b8f3d12c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:44:40.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba497468-3af4-4831-bb31-b8c7b8f3d12c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3ca8ef02 2021-08-11T16:44:40.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba497468-3af4-4831-bb31-b8c7b8f3d12c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:44:40.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba497468-3af4-4831-bb31-b8c7b8f3d12c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:44:43.634Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52af948f-0bc3-482d-8f7c-d97f0a561120 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:44:43.635Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52af948f-0bc3-482d-8f7c-d97f0a561120 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6619e79e 2021-08-11T16:44:43.635Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52af948f-0bc3-482d-8f7c-d97f0a561120 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:44:43.635Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=52af948f-0bc3-482d-8f7c-d97f0a561120 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:44:46.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3235284e-e3a1-4d9c-9b7e-c911e8f900a1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:44:46.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3235284e-e3a1-4d9c-9b7e-c911e8f900a1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@32b77e4a 2021-08-11T16:44:46.627Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3235284e-e3a1-4d9c-9b7e-c911e8f900a1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:44:46.627Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3235284e-e3a1-4d9c-9b7e-c911e8f900a1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:44:48.309Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:44:48.317Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:44:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:44:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:44:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:44:50.458Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0d4591f-049c-46b7-8277-56f326efdd5d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:44:50.459Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0d4591f-049c-46b7-8277-56f326efdd5d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@18c2b5c6 2021-08-11T16:44:50.459Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0d4591f-049c-46b7-8277-56f326efdd5d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:44:50.459Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0d4591f-049c-46b7-8277-56f326efdd5d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:44:53.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6612b39-1f17-4fa6-93aa-07d670f5530b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:44:53.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6612b39-1f17-4fa6-93aa-07d670f5530b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@63698923 2021-08-11T16:44:53.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6612b39-1f17-4fa6-93aa-07d670f5530b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:44:53.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6612b39-1f17-4fa6-93aa-07d670f5530b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:44:56.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ddb2557-30b1-4371-9940-58d857c27e5b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:44:56.654Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ddb2557-30b1-4371-9940-58d857c27e5b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4a65a7c7 2021-08-11T16:44:56.654Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ddb2557-30b1-4371-9940-58d857c27e5b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:44:56.654Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ddb2557-30b1-4371-9940-58d857c27e5b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:44:59.358Z [cluster1-nio-worker-1] WARN c.d.driver.core.HostConnectionPool Error creating connection to cassandra.onap/10.242.184.131:9042 com.datastax.driver.core.exceptions.TransportException: [cassandra.onap/10.242.184.131:9042] Connection has been closed at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1427) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1408) at com.datastax.driver.core.Connection.defunct(Connection.java:585) at com.datastax.driver.core.Connection$Dispatcher.exceptionCaught(Connection.java:1330) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273) at io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.handleReadException(AbstractEpollStreamChannel.java:728) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:818) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-08-11T16:44:59.358Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.184.131:9042 com.datastax.driver.core.exceptions.ConnectionException: [cassandra.onap/10.242.184.131:9042] Pool was closed during initialization at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:173) at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:158) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1080) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.google.common.util.concurrent.CollectionFuture.handleAllCompleted(CollectionFuture.java:69) at com.google.common.util.concurrent.AggregateFuture.processCompleted(AggregateFuture.java:282) at com.google.common.util.concurrent.AggregateFuture.decrementCountAndMaybeComplete(AggregateFuture.java:264) at com.google.common.util.concurrent.AggregateFuture.access$200(AggregateFuture.java:42) at com.google.common.util.concurrent.AggregateFuture$1.run(AggregateFuture.java:147) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:100) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.datastax.driver.core.Connection$Future.onException(Connection.java:1518) at com.datastax.driver.core.Connection$Dispatcher.errorOutAllHandler(Connection.java:1340) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1427) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1408) at com.datastax.driver.core.Connection.defunct(Connection.java:585) at com.datastax.driver.core.Connection$Dispatcher.exceptionCaught(Connection.java:1330) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273) at io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.handleReadException(AbstractEpollStreamChannel.java:728) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:818) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-08-11T16:45:00.268Z [cluster1-nio-worker-1] WARN c.d.driver.core.HostConnectionPool Error creating connection to cassandra.onap/10.242.110.137:9042 com.datastax.driver.core.exceptions.TransportException: [cassandra.onap/10.242.110.137:9042] Connection has been closed at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1427) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1408) at com.datastax.driver.core.Connection.defunct(Connection.java:585) at com.datastax.driver.core.Connection$Dispatcher.exceptionCaught(Connection.java:1330) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273) at io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.handleReadException(AbstractEpollStreamChannel.java:728) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:818) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-08-11T16:45:00.268Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.110.137:9042 com.datastax.driver.core.exceptions.ConnectionException: [cassandra.onap/10.242.110.137:9042] Pool was closed during initialization at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:173) at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:158) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1080) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.google.common.util.concurrent.CollectionFuture.handleAllCompleted(CollectionFuture.java:69) at com.google.common.util.concurrent.AggregateFuture.processCompleted(AggregateFuture.java:282) at com.google.common.util.concurrent.AggregateFuture.decrementCountAndMaybeComplete(AggregateFuture.java:264) at com.google.common.util.concurrent.AggregateFuture.access$200(AggregateFuture.java:42) at com.google.common.util.concurrent.AggregateFuture$1.run(AggregateFuture.java:147) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:100) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.datastax.driver.core.Connection$Future.onException(Connection.java:1518) at com.datastax.driver.core.Connection$Dispatcher.errorOutAllHandler(Connection.java:1340) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1427) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1408) at com.datastax.driver.core.Connection.defunct(Connection.java:585) at com.datastax.driver.core.Connection$Dispatcher.exceptionCaught(Connection.java:1330) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273) at io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.handleReadException(AbstractEpollStreamChannel.java:728) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:818) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-08-11T16:45:00.269Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c53d3e0-06d3-4ae4-b9f4-a659464f1706 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:45:00.269Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c53d3e0-06d3-4ae4-b9f4-a659464f1706 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4329d5ef 2021-08-11T16:45:00.269Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c53d3e0-06d3-4ae4-b9f4-a659464f1706 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:45:00.269Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c53d3e0-06d3-4ae4-b9f4-a659464f1706 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:45:00.287Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=6c53d3e0-06d3-4ae4-b9f4-a659464f1706 ErrorCategory=INFO ServiceName=BE Component CASSANDRA state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component CASSANDRA state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:45:00.287Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=6c53d3e0-06d3-4ae4-b9f4-a659464f1706 ErrorCategory=ERROR ServiceName=BE Component CASSANDRA state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-08-11T16:45:01.587Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85d640be-1950-4930-81e6-7cf2413f081e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:45:01.587Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85d640be-1950-4930-81e6-7cf2413f081e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4ec824c9 2021-08-11T16:45:01.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85d640be-1950-4930-81e6-7cf2413f081e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:45:01.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85d640be-1950-4930-81e6-7cf2413f081e PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:45:02.249Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:02.302Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:02.348Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:02.375Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:02.404Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:02.427Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:02.507Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:02.530Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:02.610Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:02.733Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:02.812Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:02.935Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:03.017Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:03.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:45:03.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-08-11T16:45:03.305Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:45:03.309Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:45:03.337Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:03.419Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:03.740Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:03.822Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:04.268Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:45:04.543Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:04.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2009fb51-5f10-4f01-8f60-8dad6e2c6d66 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:45:04.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2009fb51-5f10-4f01-8f60-8dad6e2c6d66 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@61479ca2 2021-08-11T16:45:04.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2009fb51-5f10-4f01-8f60-8dad6e2c6d66 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:45:04.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2009fb51-5f10-4f01-8f60-8dad6e2c6d66 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:45:04.605Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=2009fb51-5f10-4f01-8f60-8dad6e2c6d66 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:45:04.605Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=2009fb51-5f10-4f01-8f60-8dad6e2c6d66 ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-08-11T16:45:04.627Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:05.429Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:06.146Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T16:44:56.763141Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:06.232Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:06.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:45:07.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bae9cdd7-1a6d-4fef-99e4-3d3dd9f78f5c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:45:07.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bae9cdd7-1a6d-4fef-99e4-3d3dd9f78f5c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6fb44d2 2021-08-11T16:45:07.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bae9cdd7-1a6d-4fef-99e4-3d3dd9f78f5c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:45:07.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bae9cdd7-1a6d-4fef-99e4-3d3dd9f78f5c PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:45:07.834Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:09.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:45:09.437Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:10.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b4fe244-1c2f-423e-be4a-f09dbd561ea8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:45:10.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b4fe244-1c2f-423e-be4a-f09dbd561ea8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@bf06f4f 2021-08-11T16:45:10.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b4fe244-1c2f-423e-be4a-f09dbd561ea8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:45:10.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b4fe244-1c2f-423e-be4a-f09dbd561ea8 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:45:11.050Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:527) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:11.080Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:11.133Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:11.148Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:11.200Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:11.235Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:11.305Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:527) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:11.408Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:11.437Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:11.510Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:11.639Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:11.713Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:11.916Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:12.042Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:12.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:45:12.318Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:12.445Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:13.122Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:13.248Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:13.587Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=07fc3f0a-fa3c-4054-a277-e92a64cd6552 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:45:13.587Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=07fc3f0a-fa3c-4054-a277-e92a64cd6552 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@411fc94 2021-08-11T16:45:13.587Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=07fc3f0a-fa3c-4054-a277-e92a64cd6552 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:45:13.587Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=07fc3f0a-fa3c-4054-a277-e92a64cd6552 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:45:13.926Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T16:44:56.763141Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:527) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:14.051Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:15.255Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:45:15.653Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:16.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a28c2033-c07f-472a-8d85-5bc1f0f0789f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:45:16.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a28c2033-c07f-472a-8d85-5bc1f0f0789f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@596ec238 2021-08-11T16:45:16.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a28c2033-c07f-472a-8d85-5bc1f0f0789f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:45:16.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a28c2033-c07f-472a-8d85-5bc1f0f0789f PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:45:18.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:45:18.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:45:18.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:45:18.858Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:18.885Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:18.912Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:18.928Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:18.940Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:18.956Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:18.967Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:18.995Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:19.008Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:19.047Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:19.099Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:19.111Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:19.152Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:19.214Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:19.255Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:19.417Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:19.457Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:19.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=046bf237-ce0d-4f3b-b79a-5ff4255ca120 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:45:19.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=046bf237-ce0d-4f3b-b79a-5ff4255ca120 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@56fc013 2021-08-11T16:45:19.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=046bf237-ce0d-4f3b-b79a-5ff4255ca120 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:45:19.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=046bf237-ce0d-4f3b-b79a-5ff4255ca120 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:45:19.660Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:19.820Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:19.862Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:20.065Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:20.223Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:20.268Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:20.470Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:20.625Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:20.672Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:21.028Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:21.076Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:21.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:45:21.430Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:21.478Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:22.233Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T16:44:56.763141Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:22.280Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:22.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=08bc015c-3fa1-4cfb-9ea5-72237319dcfd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:45:22.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=08bc015c-3fa1-4cfb-9ea5-72237319dcfd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@ded4de6 2021-08-11T16:45:22.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=08bc015c-3fa1-4cfb-9ea5-72237319dcfd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:45:22.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=08bc015c-3fa1-4cfb-9ea5-72237319dcfd PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:45:23.883Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:24.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:45:25.652Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4397962f-d12e-459b-badb-54a804386a83 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:45:25.652Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4397962f-d12e-459b-badb-54a804386a83 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@613f593 2021-08-11T16:45:25.652Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4397962f-d12e-459b-badb-54a804386a83 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:45:25.652Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4397962f-d12e-459b-badb-54a804386a83 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:45:27.149Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:27.201Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:27.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:45:27.247Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:27.348Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:308) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:422) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:27.348Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:27.451Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:308) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:422) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:27.451Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:27.554Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:308) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:422) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:27.554Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:27.657Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:27.756Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:27.759Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:27.962Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:28.158Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:28.164Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:28.367Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:28.561Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:28.771Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:28.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96c7d0d7-273a-4476-888c-2495141eeb2d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:45:28.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96c7d0d7-273a-4476-888c-2495141eeb2d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2943986b 2021-08-11T16:45:28.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96c7d0d7-273a-4476-888c-2495141eeb2d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:45:28.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96c7d0d7-273a-4476-888c-2495141eeb2d PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:45:28.964Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:29.574Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:29.767Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:30.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:45:30.379Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T16:44:56.763141Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:31.370Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:31.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a633723d-8412-4da4-9857-1da827167434 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:45:31.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a633723d-8412-4da4-9857-1da827167434 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@52d7ba08 2021-08-11T16:45:31.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a633723d-8412-4da4-9857-1da827167434 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:45:31.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a633723d-8412-4da4-9857-1da827167434 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:45:32.972Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:33.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:45:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:45:33.311Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:45:34.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3eebf0b1-f77a-417f-8a9d-5cb1ba9945a8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:45:34.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3eebf0b1-f77a-417f-8a9d-5cb1ba9945a8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@35b24fc9 2021-08-11T16:45:34.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3eebf0b1-f77a-417f-8a9d-5cb1ba9945a8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:45:34.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3eebf0b1-f77a-417f-8a9d-5cb1ba9945a8 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:45:35.381Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:35.410Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:35.437Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:35.464Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:35.491Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:35.518Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:35.545Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:35.597Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:35.699Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:35.801Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:36.003Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:36.177Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:36.229Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:36.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:45:36.331Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:36.405Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:36.434Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:36.636Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:37.040Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:37.208Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:37.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec0bcc68-8bf2-4c64-af92-bcf7b51c1f12 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:45:37.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec0bcc68-8bf2-4c64-af92-bcf7b51c1f12 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2c148b81 2021-08-11T16:45:37.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec0bcc68-8bf2-4c64-af92-bcf7b51c1f12 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:45:37.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ec0bcc68-8bf2-4c64-af92-bcf7b51c1f12 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:45:37.847Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:38.650Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:38.813Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T16:44:56.763141Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:39.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:45:40.252Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:40.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf962ec5-6065-4c8e-81da-691d758a466c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:45:40.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf962ec5-6065-4c8e-81da-691d758a466c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1921dea8 2021-08-11T16:45:40.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf962ec5-6065-4c8e-81da-691d758a466c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:45:40.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf962ec5-6065-4c8e-81da-691d758a466c PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:45:42.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:45:43.456Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:43.508Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:43.563Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:43.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58af5710-3602-4bca-94fe-3bfb09eede0e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:45:43.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58af5710-3602-4bca-94fe-3bfb09eede0e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@76c9207e 2021-08-11T16:45:43.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58af5710-3602-4bca-94fe-3bfb09eede0e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:45:43.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58af5710-3602-4bca-94fe-3bfb09eede0e PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:45:43.620Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:43.723Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:43.816Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:43.826Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:43.868Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:43.921Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:43.928Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:43.974Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:44.026Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:44.129Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:44.133Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:44.331Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:44.335Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:44.538Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:44.735Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:44.940Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:45:45.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:45:46.608Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=22478760-a4ac-4b75-8c83-6cae8a8b58cc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:45:46.608Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=22478760-a4ac-4b75-8c83-6cae8a8b58cc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5122f569 2021-08-11T16:45:46.608Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=22478760-a4ac-4b75-8c83-6cae8a8b58cc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:45:46.608Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=22478760-a4ac-4b75-8c83-6cae8a8b58cc PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:45:47.068Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=22478760-a4ac-4b75-8c83-6cae8a8b58cc ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:45:47.068Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=22478760-a4ac-4b75-8c83-6cae8a8b58cc ErrorCategory=ERROR ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-08-11T16:45:47.270Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-08-11T16:45:48.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:45:48.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:45:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:45:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:45:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:45:49.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f009c40b-65c9-4a74-8bba-933c5cf21dd6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:45:49.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f009c40b-65c9-4a74-8bba-933c5cf21dd6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5962244f 2021-08-11T16:45:49.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f009c40b-65c9-4a74-8bba-933c5cf21dd6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:45:49.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f009c40b-65c9-4a74-8bba-933c5cf21dd6 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:45:49.967Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=f009c40b-65c9-4a74-8bba-933c5cf21dd6 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:45:49.967Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=f009c40b-65c9-4a74-8bba-933c5cf21dd6 ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-08-11T16:45:52.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37f7f4a7-db1d-4425-9f3b-8be9cfe2214f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:45:52.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37f7f4a7-db1d-4425-9f3b-8be9cfe2214f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4ea17555 2021-08-11T16:45:52.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37f7f4a7-db1d-4425-9f3b-8be9cfe2214f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:45:52.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37f7f4a7-db1d-4425-9f3b-8be9cfe2214f PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:45:55.586Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86bd603f-8ed0-4795-86f2-62c4bb1d4316 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:45:55.586Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86bd603f-8ed0-4795-86f2-62c4bb1d4316 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@71d4762f 2021-08-11T16:45:55.586Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86bd603f-8ed0-4795-86f2-62c4bb1d4316 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:45:55.587Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86bd603f-8ed0-4795-86f2-62c4bb1d4316 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:45:58.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0613e9cc-f8c4-44e5-9008-5731211aed91 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:45:58.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0613e9cc-f8c4-44e5-9008-5731211aed91 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@379b4cc1 2021-08-11T16:45:58.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0613e9cc-f8c4-44e5-9008-5731211aed91 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:45:58.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0613e9cc-f8c4-44e5-9008-5731211aed91 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:46:03.055Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb3ad9ae-7679-41e7-a69b-5036223c8ddd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:46:03.055Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb3ad9ae-7679-41e7-a69b-5036223c8ddd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7f134621 2021-08-11T16:46:03.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb3ad9ae-7679-41e7-a69b-5036223c8ddd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:46:03.056Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb3ad9ae-7679-41e7-a69b-5036223c8ddd PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:46:03.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:46:03.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:46:04.254Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:46:04.673Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=cb3ad9ae-7679-41e7-a69b-5036223c8ddd ErrorCategory=INFO ServiceName=BE Component CASSANDRA state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component CASSANDRA state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:46:04.673Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=cb3ad9ae-7679-41e7-a69b-5036223c8ddd ErrorCategory=INFO ServiceName=BE Component CASSANDRA state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:46:05.474Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a6aa1084-a6ca-49ec-8d49-643b04ac6724 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:46:05.474Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a6aa1084-a6ca-49ec-8d49-643b04ac6724 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3add9f4b 2021-08-11T16:46:05.474Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a6aa1084-a6ca-49ec-8d49-643b04ac6724 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:46:05.474Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a6aa1084-a6ca-49ec-8d49-643b04ac6724 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:46:07.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b201c088-c29e-4cb3-b61f-d60aea415cf2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:46:07.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b201c088-c29e-4cb3-b61f-d60aea415cf2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@686ae331 2021-08-11T16:46:07.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b201c088-c29e-4cb3-b61f-d60aea415cf2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:46:07.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b201c088-c29e-4cb3-b61f-d60aea415cf2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:46:10.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bfa39b1b-9939-4b06-8e43-1c376e92b2b2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:46:10.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bfa39b1b-9939-4b06-8e43-1c376e92b2b2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5fa57b9 2021-08-11T16:46:10.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bfa39b1b-9939-4b06-8e43-1c376e92b2b2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:46:10.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bfa39b1b-9939-4b06-8e43-1c376e92b2b2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:46:13.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b9805361-3c0c-48a7-9633-3cf6a312dad8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:46:13.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b9805361-3c0c-48a7-9633-3cf6a312dad8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@603d3b69 2021-08-11T16:46:13.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b9805361-3c0c-48a7-9633-3cf6a312dad8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:46:13.970Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b9805361-3c0c-48a7-9633-3cf6a312dad8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:46:16.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=22a674eb-7562-48f1-8a4f-43f20d3109cd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:46:16.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=22a674eb-7562-48f1-8a4f-43f20d3109cd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@364cc7e2 2021-08-11T16:46:16.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=22a674eb-7562-48f1-8a4f-43f20d3109cd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:46:16.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=22a674eb-7562-48f1-8a4f-43f20d3109cd PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:46:18.309Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:46:18.316Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:46:19.693Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=edae5ad7-4256-40b6-aa52-5606ab38f0d9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:46:19.693Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=edae5ad7-4256-40b6-aa52-5606ab38f0d9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@900c971 2021-08-11T16:46:19.693Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=edae5ad7-4256-40b6-aa52-5606ab38f0d9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:46:19.693Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=edae5ad7-4256-40b6-aa52-5606ab38f0d9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:46:22.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=869de499-e41e-47d2-b2d0-c77509dcd287 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:46:22.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=869de499-e41e-47d2-b2d0-c77509dcd287 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5e900077 2021-08-11T16:46:22.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=869de499-e41e-47d2-b2d0-c77509dcd287 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:46:22.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=869de499-e41e-47d2-b2d0-c77509dcd287 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:46:25.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a4a2ca45-b611-4d55-94d7-d81da9feff1d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:46:25.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a4a2ca45-b611-4d55-94d7-d81da9feff1d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@577f9ebf 2021-08-11T16:46:25.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a4a2ca45-b611-4d55-94d7-d81da9feff1d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:46:25.777Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a4a2ca45-b611-4d55-94d7-d81da9feff1d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:46:28.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a884a9b6-1156-43ff-b0cd-7559dc78e76a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:46:28.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a884a9b6-1156-43ff-b0cd-7559dc78e76a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7dc3065 2021-08-11T16:46:28.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a884a9b6-1156-43ff-b0cd-7559dc78e76a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:46:28.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a884a9b6-1156-43ff-b0cd-7559dc78e76a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:46:29.316Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01gvfb5qztoeazeaf4zvlx7xip85 2021-08-11T16:46:29.316Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01uaqxdqibfodw135p52h0ypmp1148 2021-08-11T16:46:29.317Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01ili1zbcq1g3mu7q8rvsekx4a86 2021-08-11T16:46:29.317Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node06wnqr5kupys7fzupc2d1ghn107 2021-08-11T16:46:29.317Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0ynq6v1aqodf9t44n9nqcmih066 2021-08-11T16:46:29.317Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0mqq6ll0gu62p18xtzcz3iqsd446 2021-08-11T16:46:29.317Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node015w1hh2dq9ppw17878y3surft8137 2021-08-11T16:46:29.317Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01j8r500h7501vexkc308q469i149 2021-08-11T16:46:29.317Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0ucykfjtk7xg1s9mf7oz7nzes64 2021-08-11T16:46:29.317Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0lxojuiv2so6yvr48v7x8vrp831 2021-08-11T16:46:29.317Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01owjv8ru15c771uo66xwnfmtpq114 2021-08-11T16:46:29.317Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0s0nurkjflsbtw3sy9s62q62w161 2021-08-11T16:46:29.317Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0lmes5h6ne9y4210qyj1b4ltt28 2021-08-11T16:46:29.317Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01un99htki5yppirb0pc0il8lj163 2021-08-11T16:46:29.317Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0j43cbn8ijhce1tomanittc9z383 2021-08-11T16:46:29.318Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01d40iuk3iqr2a36olhnvm65393 2021-08-11T16:46:29.318Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node030lrcnstnqxn11cykpa1cr9tb132 2021-08-11T16:46:29.318Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01mgl9p9u3jvb412fjh8d2mli6944 2021-08-11T16:46:29.318Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01a0y0pmtwjimd4tmasocgzn7q143 2021-08-11T16:46:29.318Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01l4mawqvuezhv133xxgb31ozjc69 2021-08-11T16:46:29.318Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0osbx2wcu4awu1gfi8sw350tk6125 2021-08-11T16:46:29.318Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node012qjc5i35f13m1f5939oiftb9y130 2021-08-11T16:46:29.318Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01m06hz81yz3vq1kfambvg9py9u136 2021-08-11T16:46:29.318Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0hsmnylxq55rostalrkeyr3qu27 2021-08-11T16:46:29.318Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0ymz7rvigxuw7izxq2x9qdez649 2021-08-11T16:46:29.318Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0jrx3aqbkveez1q7wdx2miaqqy119 2021-08-11T16:46:29.318Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0frw3bszna3v89pgdy4tools571 2021-08-11T16:46:29.319Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0vr2xn6su33m51xh6grnoi5465120 2021-08-11T16:46:29.319Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0rhz0p8ej9fq71045e76torr9i162 2021-08-11T16:46:29.319Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01dqpqss2hoidphpxx251cpo5w33 2021-08-11T16:46:29.319Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01g3zwe3p6c43yczg7tzsfyzzw87 2021-08-11T16:46:29.319Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0p4bfwahndgfy12f7gyttubb9b126 2021-08-11T16:46:29.319Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node09mhj4s41unmx9yl0vrlvdk6g157 2021-08-11T16:46:29.319Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01n6ue80z5sqgr1qyg9ke1etrzw92 2021-08-11T16:46:29.319Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0u3wrvm93mfjig437jfv6x6oc133 2021-08-11T16:46:29.319Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0xkd0etkw246gy073fop0j8tl116 2021-08-11T16:46:29.319Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0ffqz4gbyljaj1dc99icmtuv7z75 2021-08-11T16:46:29.319Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0gupm7kq98kjo1as50c1gub9s438 2021-08-11T16:46:29.319Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0vrlhg0xxhzq11wwqj6me2u4ch53 2021-08-11T16:46:29.319Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0kxyi3vh3ig01tbumlnvp33lr68 2021-08-11T16:46:29.319Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node012rqd6k1kx7xd7k2klfjd0zbs127 2021-08-11T16:46:29.319Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01j4arbz7mq25b9k0mxl4l6j4g139 2021-08-11T16:46:29.319Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0yu372naar7dq1dlv8wb5lp849167 2021-08-11T16:46:29.319Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0zxrhhccv0t48a6vci5a1gubm95 2021-08-11T16:46:29.320Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0elykfs8dv6wm1y10ay2vxjnkj144 2021-08-11T16:46:29.320Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node03mqykj95tm61165eqft9bxlh32 2021-08-11T16:46:29.320Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node011r9v78awyvhy1tifsmb6q9aox43 2021-08-11T16:46:29.320Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01mewmrum634cpsjd49yxw4noy160 2021-08-11T16:46:29.320Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01rkg8u2pk2zx3nkucposml3vi151 2021-08-11T16:46:29.320Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0yr46gn648l411tqxe41xhozrq58 2021-08-11T16:46:29.320Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0mi2hsuc5dpblrcw6fetg5b2b165 2021-08-11T16:46:29.320Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0os6r2rj15134dryihqgo9ww976 2021-08-11T16:46:29.320Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node025h5l1bfqaps1jpsf3mmr502h60 2021-08-11T16:46:29.325Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0zzfrnb0h1ewzag9wq8q1g05f30 2021-08-11T16:46:29.326Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01gdbh56atmt9j1pvwcn38o808a88 2021-08-11T16:46:29.326Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node03r0h236hqvs111rf9tk8x537z113 2021-08-11T16:46:29.326Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0k3clelekq4q6isnkjo54ngmm141 2021-08-11T16:46:29.326Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01j3aea1atroba18wjs1pxb69qh150 2021-08-11T16:46:29.326Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01rdxgq6nl90my1nnbfjy2nqnse45 2021-08-11T16:46:29.326Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0194ue6n2tw1imk2ngu7x7purg112 2021-08-11T16:46:29.326Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node013rj3ktolh7mkht0vwg27kjbu138 2021-08-11T16:46:29.326Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01o50vea40p0fyi2pfk6j84jx652 2021-08-11T16:46:29.326Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01sqo9g5of8g85uoi5qr3blln105 2021-08-11T16:46:29.327Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01ul1lyazod7mb1xiijlm37kbu56 2021-08-11T16:46:29.327Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01egkwvlwr26nk1lpcokjtaqlp8131 2021-08-11T16:46:29.327Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01niuwh1j5b6a9nz3rgsjapmxm134 2021-08-11T16:46:29.327Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node039wcsgawn3hy5mn4f194fqrj110 2021-08-11T16:46:29.327Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01799beakuzsuq1qfim3buyzyna123 2021-08-11T16:46:29.327Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01g8j0b4937p7z11rhpjv68kik948 2021-08-11T16:46:29.327Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node014phzdddd5fx11qwc5aaxd7mdl101 2021-08-11T16:46:29.327Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01cf2zpqjvve8mmenu0m857hb279 2021-08-11T16:46:29.327Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01hgqvt9e6ir5oltprjfbr1id5140 2021-08-11T16:46:29.327Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node025nn0rw9n1ih123nf182gbpik37 2021-08-11T16:46:29.327Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node015whg3bormqddfptjvovxxrc559 2021-08-11T16:46:29.327Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node010ji5o8mizq1c1tfbgzvqqeeqr39 2021-08-11T16:46:29.327Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01kvb45ayklbhj1wnpdy0nhvlfo91 2021-08-11T16:46:29.327Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01ovruh576onskkxklhexnjql163 2021-08-11T16:46:29.328Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0gayrm06xd4d8cjnvlngmwv5p98 2021-08-11T16:46:29.328Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01urgvev9mhn3liqo46r9z9ste55 2021-08-11T16:46:29.328Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0y3vw7c0tw5xg2zt8h9874q8d118 2021-08-11T16:46:29.328Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01xs4v93hrh8c112k9bf4tp2fkl104 2021-08-11T16:46:29.328Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01kk18gy6pjewgqiexdzasmhfg135 2021-08-11T16:46:29.328Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01kwjalxv15dboch19c1dwg7yv158 2021-08-11T16:46:29.328Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01uy8guc7exzxoc0e1yk3cxpm397 2021-08-11T16:46:29.328Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01n0y5dhbpefqjk2cshg7mh97y67 2021-08-11T16:46:29.328Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0jmatrubc4tki1lv2rqa8ndksx156 2021-08-11T16:46:29.328Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0sssn16zev5ovvy883gtfxz9t102 2021-08-11T16:46:29.328Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0eau208s8a5ll1ufhizlr8bqap50 2021-08-11T16:46:29.328Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0f9h8ooow9cnxrlp23v3zotkc111 2021-08-11T16:46:29.328Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node08sfcz859y7o418f9db0zcq1kp41 2021-08-11T16:46:29.328Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01k4ott4p3io9p1uqjr12wta91d153 2021-08-11T16:46:29.328Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node02x0ov46vkzffa9z7su7it9gj155 2021-08-11T16:46:29.329Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0ui5911evbq6fd26jbfxyyrxj25 2021-08-11T16:46:29.329Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0r0lzoghh2z8l1b90lkx8502nr35 2021-08-11T16:46:29.329Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0gak3w3kbaebg1i8vuy7uwhhzd54 2021-08-11T16:46:29.329Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0ky08z47nyxn016c62x6b6it3159 2021-08-11T16:46:29.329Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01lqdrpr2chznoom4d95n1op3h103 2021-08-11T16:46:29.329Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0lrc68i0hv044sconryrgzunx96 2021-08-11T16:46:29.329Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node05v3suv2tzvib1g3b56l86umy0121 2021-08-11T16:46:29.329Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01k5nxe287hz8z7ukhak2x8nrd36 2021-08-11T16:46:29.329Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0tm55yodwfu4dr2llshfhmna1100 2021-08-11T16:46:29.329Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0b5m8b8nfb573bd0c766yyhwl34 2021-08-11T16:46:29.329Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0wnwpqr5d5mffghk9q6zn258d166 2021-08-11T16:46:29.329Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01r1n2ziprhk9f1t2j3ol6oerh394 2021-08-11T16:46:29.329Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01q3p8chbqvm89jaiy7j0sxt6w82 2021-08-11T16:46:29.329Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0az46pg2460tpgxbxh706wuog124 2021-08-11T16:46:29.330Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0juwy6zp1x51gjbodc2ycbszy142 2021-08-11T16:46:29.330Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01eoc9fbrh9fx891g6vvui6bml40 2021-08-11T16:46:29.330Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0mm6syumakejk1be795wuz0suc47 2021-08-11T16:46:29.330Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0epqxt5eb7x5x1fjnowro8a2sn164 2021-08-11T16:46:29.330Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01jihhkn9cmf5q1h138zb193ij189 2021-08-11T16:46:29.330Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0mlc5u1ftce411pnvulk7ggype109 2021-08-11T16:46:29.330Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01xc4x8etkbyxw1b9tpw39hywks147 2021-08-11T16:46:29.330Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0b4pnjyrtpamj1ikpyhyotz7k857 2021-08-11T16:46:29.330Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0ov07jqxjqrlpej4aagrf9kne74 2021-08-11T16:46:29.330Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0upb0bgny3xeayhrwbmi6xwek168 2021-08-11T16:46:29.330Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0s43d4vyy79v9jd21s0slw7084 2021-08-11T16:46:29.330Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node011rs1bo759bv6m0lbmy93lv26146 2021-08-11T16:46:29.330Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0i6aesj4pvowt1jhn1p2vaph66108 2021-08-11T16:46:29.330Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01oqtbi61txnfnd1p1btr6e3gi90 2021-08-11T16:46:29.330Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01dyxx0n4h95911qj0ovkusnsyy145 2021-08-11T16:46:29.330Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node06pvazydxjl4kt9bcdgctk8u4154 2021-08-11T16:46:29.330Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0jky368lnmiqilflfnfjhycpk128 2021-08-11T16:46:29.331Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01lx60xsap8waq1io7wo7p7905v65 2021-08-11T16:46:29.331Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0bn94vcv782h1ayqmf33s3wso77 2021-08-11T16:46:29.331Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01oyghwo6yyxa91xzc2t3w7v3xz115 2021-08-11T16:46:29.331Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0pxw7l4wf8t5uh1zy22qsz01599 2021-08-11T16:46:29.331Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01l7jb0j6ofmpr1pprzktq8657873 2021-08-11T16:46:29.331Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0uzp8bom81owg44g0v175zfh278 2021-08-11T16:46:29.331Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0zmnuy3727ayet1v5in9llc0o51 2021-08-11T16:46:29.331Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01jzwpmnld3v781qkou66h0y94026 2021-08-11T16:46:29.331Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01a3u3deschcr41jt8re0v5tjw061 2021-08-11T16:46:29.331Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01qgsu2kah20cm120wz4b1x4b3q117 2021-08-11T16:46:29.331Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node05i0r71pi7qd1t323dq7z5lcq42 2021-08-11T16:46:29.331Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0yvpjnw1373n81h5ct2mam640929 2021-08-11T16:46:29.331Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0o57h9ydvjh5v1ik55tgq2lov5152 2021-08-11T16:46:29.331Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node03rlcgji158q21k150623xapen106 2021-08-11T16:46:29.331Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0t4k220w6nkpr1eohkm2obu51281 2021-08-11T16:46:29.331Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01yz5ke2ym4xu1a58ashm27ldu70 2021-08-11T16:46:29.332Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0l46p2n1fed1u14zvk599x78q80 2021-08-11T16:46:29.332Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01ed091q9yc8ip5uerj1oi28m5129 2021-08-11T16:46:29.332Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01q67ysx9ksahfqr9uqgg9dhtz72 2021-08-11T16:46:29.332Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0kb6neygbknqp1jr1upk5xt023122 2021-08-11T16:46:29.332Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01lkynvqtg09ul19xgc1xi34z9b62 2021-08-11T16:46:31.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=81b09b7b-b486-4453-808f-653173c80568 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:46:31.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=81b09b7b-b486-4453-808f-653173c80568 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@8182aae 2021-08-11T16:46:31.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=81b09b7b-b486-4453-808f-653173c80568 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:46:31.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=81b09b7b-b486-4453-808f-653173c80568 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:46:33.309Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:46:33.318Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:46:34.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=649bf817-4117-41a7-9f23-0532761f2ab5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:46:34.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=649bf817-4117-41a7-9f23-0532761f2ab5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@56550062 2021-08-11T16:46:34.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=649bf817-4117-41a7-9f23-0532761f2ab5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:46:34.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=649bf817-4117-41a7-9f23-0532761f2ab5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:46:37.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=278de198-8928-4dbc-b3cb-253a050c616b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:46:37.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=278de198-8928-4dbc-b3cb-253a050c616b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7d74f671 2021-08-11T16:46:37.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=278de198-8928-4dbc-b3cb-253a050c616b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:46:37.770Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=278de198-8928-4dbc-b3cb-253a050c616b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:46:40.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55242874-f620-4de4-8368-7b31de2edb7c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:46:40.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55242874-f620-4de4-8368-7b31de2edb7c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1b4dbea2 2021-08-11T16:46:40.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55242874-f620-4de4-8368-7b31de2edb7c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:46:40.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55242874-f620-4de4-8368-7b31de2edb7c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:46:43.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=80149fcf-bb53-48a7-a275-0ca5954013b5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:46:43.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=80149fcf-bb53-48a7-a275-0ca5954013b5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@52c9aa3f 2021-08-11T16:46:43.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=80149fcf-bb53-48a7-a275-0ca5954013b5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:46:43.638Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=80149fcf-bb53-48a7-a275-0ca5954013b5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:46:46.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e47d3afe-4369-4453-ba89-9fd17322ab43 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:46:46.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e47d3afe-4369-4453-ba89-9fd17322ab43 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2ce1f614 2021-08-11T16:46:46.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e47d3afe-4369-4453-ba89-9fd17322ab43 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:46:46.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e47d3afe-4369-4453-ba89-9fd17322ab43 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:46:48.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:46:48.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:46:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:46:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:46:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:46:49.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d902beee-446a-47c0-b195-26534e5be841 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:46:49.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d902beee-446a-47c0-b195-26534e5be841 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@76fc5a87 2021-08-11T16:46:49.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d902beee-446a-47c0-b195-26534e5be841 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:46:49.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d902beee-446a-47c0-b195-26534e5be841 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:46:52.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c9a9724-c6d6-4396-ab26-20683dd77dab PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:46:52.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c9a9724-c6d6-4396-ab26-20683dd77dab PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@324254 2021-08-11T16:46:52.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c9a9724-c6d6-4396-ab26-20683dd77dab PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:46:52.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c9a9724-c6d6-4396-ab26-20683dd77dab PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:46:56.170Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=154e61ec-5c3b-4113-8a45-beeea9c441f1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:46:56.170Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=154e61ec-5c3b-4113-8a45-beeea9c441f1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1f35883f 2021-08-11T16:46:56.170Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=154e61ec-5c3b-4113-8a45-beeea9c441f1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:46:56.170Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=154e61ec-5c3b-4113-8a45-beeea9c441f1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:46:58.696Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8fb4a28e-efba-4e2b-a104-4d5685ff2382 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:46:58.696Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8fb4a28e-efba-4e2b-a104-4d5685ff2382 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3cc79c9b 2021-08-11T16:46:58.696Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8fb4a28e-efba-4e2b-a104-4d5685ff2382 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:46:58.696Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8fb4a28e-efba-4e2b-a104-4d5685ff2382 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:47:01.652Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e4e5f04-9435-4ef9-8f1f-bee1a4e142ec PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:47:01.652Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e4e5f04-9435-4ef9-8f1f-bee1a4e142ec PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@287bd1e6 2021-08-11T16:47:01.652Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e4e5f04-9435-4ef9-8f1f-bee1a4e142ec PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:47:01.652Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e4e5f04-9435-4ef9-8f1f-bee1a4e142ec PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:47:03.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:47:03.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:47:04.262Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:47:04.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a4d0a1a-6a05-4a2b-a29d-6a5c5a25f1d5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:47:04.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a4d0a1a-6a05-4a2b-a29d-6a5c5a25f1d5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3ac9065f 2021-08-11T16:47:04.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a4d0a1a-6a05-4a2b-a29d-6a5c5a25f1d5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:47:04.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a4d0a1a-6a05-4a2b-a29d-6a5c5a25f1d5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:47:07.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=00e7cea5-b623-46b8-bc84-b9858b2cd63c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:47:07.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=00e7cea5-b623-46b8-bc84-b9858b2cd63c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@98e0549 2021-08-11T16:47:07.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=00e7cea5-b623-46b8-bc84-b9858b2cd63c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:47:07.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=00e7cea5-b623-46b8-bc84-b9858b2cd63c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:47:10.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b8c5f9b2-6719-441d-8b31-cb4665ab1424 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:47:10.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b8c5f9b2-6719-441d-8b31-cb4665ab1424 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@9ad63a7 2021-08-11T16:47:10.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b8c5f9b2-6719-441d-8b31-cb4665ab1424 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:47:10.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b8c5f9b2-6719-441d-8b31-cb4665ab1424 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:47:13.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab4c0daf-a589-4d34-8a7a-6be7ea72f9f9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:47:13.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab4c0daf-a589-4d34-8a7a-6be7ea72f9f9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7f2f7795 2021-08-11T16:47:13.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab4c0daf-a589-4d34-8a7a-6be7ea72f9f9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:47:13.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab4c0daf-a589-4d34-8a7a-6be7ea72f9f9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:47:16.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9cb2338e-3e58-4df8-baa7-cd72725b77e2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:47:16.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9cb2338e-3e58-4df8-baa7-cd72725b77e2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@da67dd1 2021-08-11T16:47:16.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9cb2338e-3e58-4df8-baa7-cd72725b77e2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:47:16.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9cb2338e-3e58-4df8-baa7-cd72725b77e2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:47:18.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:47:18.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:47:19.705Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dba7d058-83ec-42ae-8c59-0f981a61f094 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:47:19.705Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dba7d058-83ec-42ae-8c59-0f981a61f094 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@747b9561 2021-08-11T16:47:19.705Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dba7d058-83ec-42ae-8c59-0f981a61f094 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:47:19.705Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dba7d058-83ec-42ae-8c59-0f981a61f094 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:47:22.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50dbf48b-87ea-4c4d-a172-f0057bcaa5ad PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:47:22.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50dbf48b-87ea-4c4d-a172-f0057bcaa5ad PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@16a97012 2021-08-11T16:47:22.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50dbf48b-87ea-4c4d-a172-f0057bcaa5ad PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:47:22.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50dbf48b-87ea-4c4d-a172-f0057bcaa5ad PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:47:25.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=646bc4da-96df-44a3-a0c1-3d6d4d11c485 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:47:25.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=646bc4da-96df-44a3-a0c1-3d6d4d11c485 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@123e3e8d 2021-08-11T16:47:25.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=646bc4da-96df-44a3-a0c1-3d6d4d11c485 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:47:25.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=646bc4da-96df-44a3-a0c1-3d6d4d11c485 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:47:28.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d110240a-a811-42eb-8690-194988001caa PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:47:28.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d110240a-a811-42eb-8690-194988001caa PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2c4881ee 2021-08-11T16:47:28.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d110240a-a811-42eb-8690-194988001caa PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:47:28.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d110240a-a811-42eb-8690-194988001caa PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:47:31.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=45e15496-c7a7-4335-a0c1-ad297674594b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:47:31.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=45e15496-c7a7-4335-a0c1-ad297674594b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@430c99b 2021-08-11T16:47:31.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=45e15496-c7a7-4335-a0c1-ad297674594b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:47:31.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=45e15496-c7a7-4335-a0c1-ad297674594b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:47:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:47:33.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:47:34.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=80c7a9d0-4419-4d8a-900e-f853085bb731 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:47:34.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=80c7a9d0-4419-4d8a-900e-f853085bb731 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@594410ef 2021-08-11T16:47:34.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=80c7a9d0-4419-4d8a-900e-f853085bb731 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:47:34.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=80c7a9d0-4419-4d8a-900e-f853085bb731 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:47:37.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d427ee97-db31-4094-bebe-0af6653bd467 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:47:37.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d427ee97-db31-4094-bebe-0af6653bd467 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3a6d971c 2021-08-11T16:47:37.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d427ee97-db31-4094-bebe-0af6653bd467 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:47:37.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d427ee97-db31-4094-bebe-0af6653bd467 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:47:40.854Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f342f45-32bb-499c-937b-7b880625d65b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:47:40.854Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f342f45-32bb-499c-937b-7b880625d65b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5242264e 2021-08-11T16:47:40.854Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f342f45-32bb-499c-937b-7b880625d65b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:47:40.854Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9f342f45-32bb-499c-937b-7b880625d65b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:47:43.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4db4ffd-7313-40ac-b9fb-1b7ad82d1d42 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:47:43.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4db4ffd-7313-40ac-b9fb-1b7ad82d1d42 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@503c418c 2021-08-11T16:47:43.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4db4ffd-7313-40ac-b9fb-1b7ad82d1d42 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:47:43.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4db4ffd-7313-40ac-b9fb-1b7ad82d1d42 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:47:46.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a047750f-cfd1-46ce-b0a9-b1c1e3893be4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:47:46.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a047750f-cfd1-46ce-b0a9-b1c1e3893be4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2906e00c 2021-08-11T16:47:46.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a047750f-cfd1-46ce-b0a9-b1c1e3893be4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:47:46.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a047750f-cfd1-46ce-b0a9-b1c1e3893be4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:47:48.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:47:48.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:47:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:47:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:47:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:47:49.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=612af19c-8f33-4925-934a-9c6efb32bb2f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:47:49.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=612af19c-8f33-4925-934a-9c6efb32bb2f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7f855747 2021-08-11T16:47:49.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=612af19c-8f33-4925-934a-9c6efb32bb2f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:47:49.957Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=612af19c-8f33-4925-934a-9c6efb32bb2f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:47:53.462Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=768f8373-781e-4e78-8d70-f695a5b821c3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:47:53.462Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=768f8373-781e-4e78-8d70-f695a5b821c3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3cb66eaf 2021-08-11T16:47:53.462Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=768f8373-781e-4e78-8d70-f695a5b821c3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:47:53.462Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=768f8373-781e-4e78-8d70-f695a5b821c3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:47:56.462Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7680ddf3-8460-4264-b9d2-01b51deda9cf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:47:56.462Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7680ddf3-8460-4264-b9d2-01b51deda9cf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@e346894 2021-08-11T16:47:56.462Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7680ddf3-8460-4264-b9d2-01b51deda9cf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:47:56.463Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7680ddf3-8460-4264-b9d2-01b51deda9cf PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:47:59.170Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0a9975d1-e913-4bd6-845a-a1cde4e58531 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:47:59.170Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0a9975d1-e913-4bd6-845a-a1cde4e58531 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@10acefde 2021-08-11T16:47:59.170Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0a9975d1-e913-4bd6-845a-a1cde4e58531 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:47:59.170Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0a9975d1-e913-4bd6-845a-a1cde4e58531 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:48:01.693Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29134fef-27ef-44a6-b14f-3d393ab174e2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:48:01.693Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29134fef-27ef-44a6-b14f-3d393ab174e2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4546ade3 2021-08-11T16:48:01.693Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29134fef-27ef-44a6-b14f-3d393ab174e2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:48:01.693Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29134fef-27ef-44a6-b14f-3d393ab174e2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:48:03.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:48:03.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:48:04.268Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:48:04.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c48bcf21-765b-4d48-a328-fc1c058cb05c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:48:04.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c48bcf21-765b-4d48-a328-fc1c058cb05c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4a8a70f2 2021-08-11T16:48:04.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c48bcf21-765b-4d48-a328-fc1c058cb05c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:48:04.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c48bcf21-765b-4d48-a328-fc1c058cb05c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:48:07.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4ecaec3-9202-401a-938a-d822153c1b0b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:48:07.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4ecaec3-9202-401a-938a-d822153c1b0b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6757c1ff 2021-08-11T16:48:07.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4ecaec3-9202-401a-938a-d822153c1b0b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:48:07.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4ecaec3-9202-401a-938a-d822153c1b0b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:48:10.632Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3325b3a-e34a-414f-8024-f0e15253aacf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:48:10.633Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3325b3a-e34a-414f-8024-f0e15253aacf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@107210d4 2021-08-11T16:48:10.633Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3325b3a-e34a-414f-8024-f0e15253aacf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:48:10.633Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3325b3a-e34a-414f-8024-f0e15253aacf PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:48:13.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e24345c0-5e4a-436f-81ef-a71dd7f29151 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:48:13.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e24345c0-5e4a-436f-81ef-a71dd7f29151 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@602b8e8c 2021-08-11T16:48:13.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e24345c0-5e4a-436f-81ef-a71dd7f29151 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:48:13.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e24345c0-5e4a-436f-81ef-a71dd7f29151 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:48:16.846Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d4312c75-8be7-499a-87c8-4a9f864ef17f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:48:16.846Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d4312c75-8be7-499a-87c8-4a9f864ef17f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@793f6f9b 2021-08-11T16:48:16.846Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d4312c75-8be7-499a-87c8-4a9f864ef17f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:48:16.846Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d4312c75-8be7-499a-87c8-4a9f864ef17f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:48:18.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:48:18.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:48:19.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=782b1dc8-ee76-419d-89ea-0b8ad236d8a9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:48:19.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=782b1dc8-ee76-419d-89ea-0b8ad236d8a9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@203e017a 2021-08-11T16:48:19.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=782b1dc8-ee76-419d-89ea-0b8ad236d8a9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:48:19.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=782b1dc8-ee76-419d-89ea-0b8ad236d8a9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:48:22.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=59a6c980-f337-472a-bb18-4a89c782e0ab PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:48:22.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=59a6c980-f337-472a-bb18-4a89c782e0ab PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@204a4119 2021-08-11T16:48:22.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=59a6c980-f337-472a-bb18-4a89c782e0ab PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:48:22.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=59a6c980-f337-472a-bb18-4a89c782e0ab PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:48:25.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=778af1ca-e252-4e78-97db-995b232a0bc1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:48:25.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=778af1ca-e252-4e78-97db-995b232a0bc1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1c29e472 2021-08-11T16:48:25.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=778af1ca-e252-4e78-97db-995b232a0bc1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:48:25.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=778af1ca-e252-4e78-97db-995b232a0bc1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:48:28.854Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d4c845ec-6ab3-43ef-ab1d-ccd1d7617001 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:48:28.854Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d4c845ec-6ab3-43ef-ab1d-ccd1d7617001 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1ee803db 2021-08-11T16:48:28.854Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d4c845ec-6ab3-43ef-ab1d-ccd1d7617001 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:48:28.854Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d4c845ec-6ab3-43ef-ab1d-ccd1d7617001 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:48:31.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=afb482c4-ede7-4797-a10f-1ae64c15d452 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:48:31.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=afb482c4-ede7-4797-a10f-1ae64c15d452 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4e0de4ad 2021-08-11T16:48:31.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=afb482c4-ede7-4797-a10f-1ae64c15d452 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:48:31.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=afb482c4-ede7-4797-a10f-1ae64c15d452 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:48:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:48:33.316Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:48:34.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2cbca3dd-d329-4bb5-a22e-96a15a5918d4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:48:34.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2cbca3dd-d329-4bb5-a22e-96a15a5918d4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3cbd36fc 2021-08-11T16:48:34.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2cbca3dd-d329-4bb5-a22e-96a15a5918d4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:48:34.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2cbca3dd-d329-4bb5-a22e-96a15a5918d4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:48:37.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24f61727-f42f-470b-bee5-b6ba489958ff PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:48:37.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24f61727-f42f-470b-bee5-b6ba489958ff PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6f644f6d 2021-08-11T16:48:37.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24f61727-f42f-470b-bee5-b6ba489958ff PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:48:37.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24f61727-f42f-470b-bee5-b6ba489958ff PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:48:40.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77cc83ea-7c1d-4a2c-87bb-b4a77e9072fe PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:48:40.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77cc83ea-7c1d-4a2c-87bb-b4a77e9072fe PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@31cc7f81 2021-08-11T16:48:40.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77cc83ea-7c1d-4a2c-87bb-b4a77e9072fe PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:48:40.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77cc83ea-7c1d-4a2c-87bb-b4a77e9072fe PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:48:43.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c6b9f2c-66bc-4609-a49b-9798eae41e31 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:48:43.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c6b9f2c-66bc-4609-a49b-9798eae41e31 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1045cd99 2021-08-11T16:48:43.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c6b9f2c-66bc-4609-a49b-9798eae41e31 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:48:43.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7c6b9f2c-66bc-4609-a49b-9798eae41e31 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:48:46.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a4914af1-410b-455e-a0f9-dc29f16167eb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:48:46.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a4914af1-410b-455e-a0f9-dc29f16167eb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@459a13bb 2021-08-11T16:48:46.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a4914af1-410b-455e-a0f9-dc29f16167eb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:48:46.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a4914af1-410b-455e-a0f9-dc29f16167eb PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:48:48.305Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:48:48.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:48:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:48:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:48:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:48:50.458Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79779358-503c-40f6-91dd-08773166d979 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:48:50.458Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79779358-503c-40f6-91dd-08773166d979 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@8483342 2021-08-11T16:48:50.458Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79779358-503c-40f6-91dd-08773166d979 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:48:50.458Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79779358-503c-40f6-91dd-08773166d979 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:48:53.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f4969b4-0a95-4e2e-af51-abd3d0e6caa5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:48:53.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f4969b4-0a95-4e2e-af51-abd3d0e6caa5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5aaa75bd 2021-08-11T16:48:53.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f4969b4-0a95-4e2e-af51-abd3d0e6caa5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:48:53.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f4969b4-0a95-4e2e-af51-abd3d0e6caa5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:48:56.451Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=273a89e9-6ad4-430d-8d34-9fca5e5b967d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:48:56.451Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=273a89e9-6ad4-430d-8d34-9fca5e5b967d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@64bd4274 2021-08-11T16:48:56.451Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=273a89e9-6ad4-430d-8d34-9fca5e5b967d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:48:56.451Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=273a89e9-6ad4-430d-8d34-9fca5e5b967d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:48:59.966Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2af16ead-78f8-4ae4-a8a4-1f507ff7a339 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:48:59.966Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2af16ead-78f8-4ae4-a8a4-1f507ff7a339 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@71af9293 2021-08-11T16:48:59.966Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2af16ead-78f8-4ae4-a8a4-1f507ff7a339 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:48:59.966Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2af16ead-78f8-4ae4-a8a4-1f507ff7a339 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:49:02.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb3f3713-be80-4513-88b6-24b6274e2616 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:49:02.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb3f3713-be80-4513-88b6-24b6274e2616 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@38347dc7 2021-08-11T16:49:02.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb3f3713-be80-4513-88b6-24b6274e2616 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:49:02.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb3f3713-be80-4513-88b6-24b6274e2616 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:49:03.305Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:49:03.310Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:49:04.254Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:49:05.375Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3032949b-14ed-4d4f-89f2-f50aef840098 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:49:05.375Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3032949b-14ed-4d4f-89f2-f50aef840098 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@31baf0e1 2021-08-11T16:49:05.375Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3032949b-14ed-4d4f-89f2-f50aef840098 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:49:05.375Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3032949b-14ed-4d4f-89f2-f50aef840098 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:49:07.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=999af186-ff2e-41ee-a5fd-72d164f8821e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:49:07.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=999af186-ff2e-41ee-a5fd-72d164f8821e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3f1aba40 2021-08-11T16:49:07.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=999af186-ff2e-41ee-a5fd-72d164f8821e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:49:07.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=999af186-ff2e-41ee-a5fd-72d164f8821e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:49:10.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0c580dcb-1a04-4e4f-ae10-bbbaca74a192 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:49:10.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0c580dcb-1a04-4e4f-ae10-bbbaca74a192 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@32ac8c3a 2021-08-11T16:49:10.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0c580dcb-1a04-4e4f-ae10-bbbaca74a192 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:49:10.780Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0c580dcb-1a04-4e4f-ae10-bbbaca74a192 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:49:13.633Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58b36cb5-056f-4f3e-a7eb-c765358e1269 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:49:13.633Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58b36cb5-056f-4f3e-a7eb-c765358e1269 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@296d05ed 2021-08-11T16:49:13.634Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58b36cb5-056f-4f3e-a7eb-c765358e1269 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:49:13.634Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=58b36cb5-056f-4f3e-a7eb-c765358e1269 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:49:16.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db5a8e55-42e3-4d64-ac92-34ff5ee92996 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:49:16.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db5a8e55-42e3-4d64-ac92-34ff5ee92996 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4fb6d5d9 2021-08-11T16:49:16.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db5a8e55-42e3-4d64-ac92-34ff5ee92996 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:49:16.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db5a8e55-42e3-4d64-ac92-34ff5ee92996 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:49:18.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:49:18.316Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:49:19.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=775cf138-8cf2-43c9-a565-3b64af997853 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:49:19.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=775cf138-8cf2-43c9-a565-3b64af997853 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2594aed5 2021-08-11T16:49:19.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=775cf138-8cf2-43c9-a565-3b64af997853 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:49:19.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=775cf138-8cf2-43c9-a565-3b64af997853 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:49:22.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=83548a03-2e71-41b2-b97b-5d40e1f27464 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:49:22.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=83548a03-2e71-41b2-b97b-5d40e1f27464 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@738f2b01 2021-08-11T16:49:22.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=83548a03-2e71-41b2-b97b-5d40e1f27464 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:49:22.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=83548a03-2e71-41b2-b97b-5d40e1f27464 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:49:25.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=00780bdd-9ef7-4c23-9ce6-adae6f15cf93 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:49:25.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=00780bdd-9ef7-4c23-9ce6-adae6f15cf93 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@320c0af0 2021-08-11T16:49:25.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=00780bdd-9ef7-4c23-9ce6-adae6f15cf93 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:49:25.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=00780bdd-9ef7-4c23-9ce6-adae6f15cf93 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:49:28.851Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd94429c-a135-46fe-aba7-d4008fd93978 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:49:28.851Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd94429c-a135-46fe-aba7-d4008fd93978 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4bed1975 2021-08-11T16:49:28.851Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd94429c-a135-46fe-aba7-d4008fd93978 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:49:28.851Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd94429c-a135-46fe-aba7-d4008fd93978 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:49:31.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9c439d45-a429-4a81-a1a9-4a7ab87647ee PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:49:31.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9c439d45-a429-4a81-a1a9-4a7ab87647ee PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4b0ff4db 2021-08-11T16:49:31.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9c439d45-a429-4a81-a1a9-4a7ab87647ee PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:49:31.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9c439d45-a429-4a81-a1a9-4a7ab87647ee PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:49:33.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:49:33.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:49:34.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7ebf642-3564-4d1e-b2a6-415ba273dfda PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:49:34.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7ebf642-3564-4d1e-b2a6-415ba273dfda PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@41d20998 2021-08-11T16:49:34.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7ebf642-3564-4d1e-b2a6-415ba273dfda PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:49:34.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7ebf642-3564-4d1e-b2a6-415ba273dfda PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:49:37.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6a97634-8b27-4b5e-a1b7-7eedd76e59e1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:49:37.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6a97634-8b27-4b5e-a1b7-7eedd76e59e1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@135bd0d8 2021-08-11T16:49:37.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6a97634-8b27-4b5e-a1b7-7eedd76e59e1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:49:37.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6a97634-8b27-4b5e-a1b7-7eedd76e59e1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:49:40.949Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8c04a3db-813d-4552-9292-d40a6a33b7f1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:49:40.949Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8c04a3db-813d-4552-9292-d40a6a33b7f1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@305e5577 2021-08-11T16:49:40.949Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8c04a3db-813d-4552-9292-d40a6a33b7f1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:49:40.950Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8c04a3db-813d-4552-9292-d40a6a33b7f1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:49:43.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3541d1f2-ec22-4d90-a2f0-daf6df888fa1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:49:43.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3541d1f2-ec22-4d90-a2f0-daf6df888fa1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4c3c68b 2021-08-11T16:49:43.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3541d1f2-ec22-4d90-a2f0-daf6df888fa1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:49:43.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3541d1f2-ec22-4d90-a2f0-daf6df888fa1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:49:46.751Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90b5d0d3-d89c-4791-bd52-523943c17e58 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:49:46.752Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90b5d0d3-d89c-4791-bd52-523943c17e58 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@558c8142 2021-08-11T16:49:46.752Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90b5d0d3-d89c-4791-bd52-523943c17e58 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:49:46.752Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90b5d0d3-d89c-4791-bd52-523943c17e58 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:49:48.309Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:49:48.317Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:49:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:49:49.292Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:49:49.292Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:49:50.259Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=102bac31-0aa4-4631-bd5f-bbdb7035fecb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:49:50.259Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=102bac31-0aa4-4631-bd5f-bbdb7035fecb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@181c463b 2021-08-11T16:49:50.259Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=102bac31-0aa4-4631-bd5f-bbdb7035fecb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:49:50.259Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=102bac31-0aa4-4631-bd5f-bbdb7035fecb PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:49:53.263Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9021247a-e6d4-48a5-8e4e-974611201f0f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:49:53.263Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9021247a-e6d4-48a5-8e4e-974611201f0f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5aab5da2 2021-08-11T16:49:53.263Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9021247a-e6d4-48a5-8e4e-974611201f0f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:49:53.263Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9021247a-e6d4-48a5-8e4e-974611201f0f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:49:56.467Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4d1ca1b3-09e6-4e29-82f0-912c31a24bee PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:49:56.467Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4d1ca1b3-09e6-4e29-82f0-912c31a24bee PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@40c63fe2 2021-08-11T16:49:56.467Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4d1ca1b3-09e6-4e29-82f0-912c31a24bee PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:49:56.467Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4d1ca1b3-09e6-4e29-82f0-912c31a24bee PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:49:59.571Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a209360d-4309-40db-91b4-8ad9a36cf115 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:49:59.571Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a209360d-4309-40db-91b4-8ad9a36cf115 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@71713029 2021-08-11T16:49:59.571Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a209360d-4309-40db-91b4-8ad9a36cf115 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:49:59.571Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a209360d-4309-40db-91b4-8ad9a36cf115 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:50:01.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21a7bcc3-c1b5-4ec8-a9f3-a6467ce74a98 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:50:01.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21a7bcc3-c1b5-4ec8-a9f3-a6467ce74a98 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@ba69da2 2021-08-11T16:50:01.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21a7bcc3-c1b5-4ec8-a9f3-a6467ce74a98 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:50:01.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21a7bcc3-c1b5-4ec8-a9f3-a6467ce74a98 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:50:03.309Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:50:03.316Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:50:04.251Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:50:04.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06e7dc5b-d136-440d-927c-25ea163b4221 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:50:04.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06e7dc5b-d136-440d-927c-25ea163b4221 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@be09739 2021-08-11T16:50:04.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06e7dc5b-d136-440d-927c-25ea163b4221 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:50:04.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06e7dc5b-d136-440d-927c-25ea163b4221 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:50:07.825Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47d157a5-e08a-48e6-894b-05f88cd47991 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:50:07.825Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47d157a5-e08a-48e6-894b-05f88cd47991 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7a288088 2021-08-11T16:50:07.825Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47d157a5-e08a-48e6-894b-05f88cd47991 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:50:07.825Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47d157a5-e08a-48e6-894b-05f88cd47991 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:50:10.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0a2be4c-3a34-4def-aa78-6c6695f66f2a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:50:10.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0a2be4c-3a34-4def-aa78-6c6695f66f2a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2a0179c 2021-08-11T16:50:10.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0a2be4c-3a34-4def-aa78-6c6695f66f2a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:50:10.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b0a2be4c-3a34-4def-aa78-6c6695f66f2a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:50:13.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a25933a-0e9e-4ba0-b2f6-a95c2aba7b9e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:50:13.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a25933a-0e9e-4ba0-b2f6-a95c2aba7b9e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3dd4c1e7 2021-08-11T16:50:13.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a25933a-0e9e-4ba0-b2f6-a95c2aba7b9e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:50:13.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a25933a-0e9e-4ba0-b2f6-a95c2aba7b9e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:50:16.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7daf6d46-3a22-4e59-9146-52495a42687e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:50:16.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7daf6d46-3a22-4e59-9146-52495a42687e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3282d03b 2021-08-11T16:50:16.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7daf6d46-3a22-4e59-9146-52495a42687e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:50:16.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7daf6d46-3a22-4e59-9146-52495a42687e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:50:18.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:50:18.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:50:19.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5bdfa81-d957-4e02-b6e9-24d6cd72061c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:50:19.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5bdfa81-d957-4e02-b6e9-24d6cd72061c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@c102a23 2021-08-11T16:50:19.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5bdfa81-d957-4e02-b6e9-24d6cd72061c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:50:19.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e5bdfa81-d957-4e02-b6e9-24d6cd72061c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:50:22.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4180c250-562a-43c1-9fe4-a9e3bd2c7747 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:50:22.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4180c250-562a-43c1-9fe4-a9e3bd2c7747 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@e8ec0ca 2021-08-11T16:50:22.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4180c250-562a-43c1-9fe4-a9e3bd2c7747 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:50:22.598Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4180c250-562a-43c1-9fe4-a9e3bd2c7747 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:50:25.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5d0b554a-570e-4053-bc6f-3111f118d449 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:50:25.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5d0b554a-570e-4053-bc6f-3111f118d449 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@154ef305 2021-08-11T16:50:25.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5d0b554a-570e-4053-bc6f-3111f118d449 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:50:25.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5d0b554a-570e-4053-bc6f-3111f118d449 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:50:28.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eeb1ef33-b3c6-469a-9447-2eee3510449a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:50:28.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eeb1ef33-b3c6-469a-9447-2eee3510449a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@31e9ea9f 2021-08-11T16:50:28.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eeb1ef33-b3c6-469a-9447-2eee3510449a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:50:28.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eeb1ef33-b3c6-469a-9447-2eee3510449a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:50:31.599Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77034429-5662-49ba-9072-eee8e803ffe6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:50:31.599Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77034429-5662-49ba-9072-eee8e803ffe6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3c259fb3 2021-08-11T16:50:31.599Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77034429-5662-49ba-9072-eee8e803ffe6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:50:31.599Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77034429-5662-49ba-9072-eee8e803ffe6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:50:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:50:33.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:50:34.612Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c5e98010-23cd-461c-b11b-9dcfe420218c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:50:34.612Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c5e98010-23cd-461c-b11b-9dcfe420218c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4455906d 2021-08-11T16:50:34.612Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c5e98010-23cd-461c-b11b-9dcfe420218c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:50:34.612Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c5e98010-23cd-461c-b11b-9dcfe420218c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:50:37.637Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06a400e1-a3df-486f-a9b8-3a197bfaa51c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:50:37.637Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06a400e1-a3df-486f-a9b8-3a197bfaa51c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@26619539 2021-08-11T16:50:37.637Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06a400e1-a3df-486f-a9b8-3a197bfaa51c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:50:37.637Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06a400e1-a3df-486f-a9b8-3a197bfaa51c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:50:40.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f16bf123-e62a-48af-a3fc-9b7c86c19473 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:50:40.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f16bf123-e62a-48af-a3fc-9b7c86c19473 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@58a22e78 2021-08-11T16:50:40.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f16bf123-e62a-48af-a3fc-9b7c86c19473 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:50:40.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f16bf123-e62a-48af-a3fc-9b7c86c19473 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:50:43.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7ebd9acb-80e4-42d6-b274-76de589a0f5d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:50:43.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7ebd9acb-80e4-42d6-b274-76de589a0f5d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@48eaead8 2021-08-11T16:50:43.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7ebd9acb-80e4-42d6-b274-76de589a0f5d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:50:43.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7ebd9acb-80e4-42d6-b274-76de589a0f5d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:50:46.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0989ef5e-8b58-44c3-92bd-a64cd1c00786 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:50:46.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0989ef5e-8b58-44c3-92bd-a64cd1c00786 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@20d4cd50 2021-08-11T16:50:46.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0989ef5e-8b58-44c3-92bd-a64cd1c00786 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:50:46.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0989ef5e-8b58-44c3-92bd-a64cd1c00786 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:50:48.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:50:48.311Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:50:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:50:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:50:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:50:50.349Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11b56bd4-77b2-45a9-973a-d632faa5e881 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:50:50.349Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11b56bd4-77b2-45a9-973a-d632faa5e881 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@ab64c2e 2021-08-11T16:50:50.349Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11b56bd4-77b2-45a9-973a-d632faa5e881 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:50:50.349Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11b56bd4-77b2-45a9-973a-d632faa5e881 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:50:53.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf7b69fc-5db7-471a-9ac4-0d7055c0203d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:50:53.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf7b69fc-5db7-471a-9ac4-0d7055c0203d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@43ae72ee 2021-08-11T16:50:53.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf7b69fc-5db7-471a-9ac4-0d7055c0203d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:50:53.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf7b69fc-5db7-471a-9ac4-0d7055c0203d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:50:56.954Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79a7c3db-9240-49f5-b5d9-d73aa633967c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:50:56.954Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79a7c3db-9240-49f5-b5d9-d73aa633967c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5c0dfc69 2021-08-11T16:50:56.954Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79a7c3db-9240-49f5-b5d9-d73aa633967c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:50:56.954Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79a7c3db-9240-49f5-b5d9-d73aa633967c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:51:00.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79d194c2-ab76-4594-bcd2-9b4300e84178 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:51:00.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79d194c2-ab76-4594-bcd2-9b4300e84178 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5ab514f5 2021-08-11T16:51:00.068Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79d194c2-ab76-4594-bcd2-9b4300e84178 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:51:00.069Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79d194c2-ab76-4594-bcd2-9b4300e84178 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:51:02.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b38c93ec-76fa-4004-b570-5ae1acab8eff PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:51:02.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b38c93ec-76fa-4004-b570-5ae1acab8eff PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@62dae4b1 2021-08-11T16:51:02.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b38c93ec-76fa-4004-b570-5ae1acab8eff PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:51:02.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b38c93ec-76fa-4004-b570-5ae1acab8eff PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:51:03.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:51:03.311Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:51:04.382Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:51:05.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c8f746b-1b82-4af9-b70b-a79fc2b4b492 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:51:05.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c8f746b-1b82-4af9-b70b-a79fc2b4b492 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5253d306 2021-08-11T16:51:05.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c8f746b-1b82-4af9-b70b-a79fc2b4b492 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:51:05.268Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c8f746b-1b82-4af9-b70b-a79fc2b4b492 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:51:07.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b25f095-5887-4bcb-bd67-133edcd4800f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:51:07.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b25f095-5887-4bcb-bd67-133edcd4800f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@610e9391 2021-08-11T16:51:07.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b25f095-5887-4bcb-bd67-133edcd4800f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:51:07.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b25f095-5887-4bcb-bd67-133edcd4800f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:51:10.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6be91391-819d-4fd5-92db-2a1ca68060d7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:51:10.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6be91391-819d-4fd5-92db-2a1ca68060d7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@136e31f9 2021-08-11T16:51:10.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6be91391-819d-4fd5-92db-2a1ca68060d7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:51:10.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6be91391-819d-4fd5-92db-2a1ca68060d7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:51:14.186Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee8966fb-599e-4884-a5d1-426118de38da PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:51:14.186Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee8966fb-599e-4884-a5d1-426118de38da PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2624a049 2021-08-11T16:51:14.186Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee8966fb-599e-4884-a5d1-426118de38da PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:51:14.186Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee8966fb-599e-4884-a5d1-426118de38da PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:51:16.599Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ca580a6c-7960-4d78-a231-e3acac4b2ff3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:51:16.599Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ca580a6c-7960-4d78-a231-e3acac4b2ff3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6aa39c68 2021-08-11T16:51:16.600Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ca580a6c-7960-4d78-a231-e3acac4b2ff3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:51:16.600Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ca580a6c-7960-4d78-a231-e3acac4b2ff3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:51:18.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:51:18.318Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:51:19.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d11ff950-e85b-4162-b64c-679184f65e97 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:51:19.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d11ff950-e85b-4162-b64c-679184f65e97 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7f2408b6 2021-08-11T16:51:19.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d11ff950-e85b-4162-b64c-679184f65e97 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:51:19.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d11ff950-e85b-4162-b64c-679184f65e97 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:51:22.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eacffa6b-c6ec-4fa0-9a03-2d4ef1d9beab PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:51:22.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eacffa6b-c6ec-4fa0-9a03-2d4ef1d9beab PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1da70f52 2021-08-11T16:51:22.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eacffa6b-c6ec-4fa0-9a03-2d4ef1d9beab PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:51:22.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eacffa6b-c6ec-4fa0-9a03-2d4ef1d9beab PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:51:25.747Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8ceb79ee-10f0-4986-a54a-1f992e713d1c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:51:25.747Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8ceb79ee-10f0-4986-a54a-1f992e713d1c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1e02d247 2021-08-11T16:51:25.747Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8ceb79ee-10f0-4986-a54a-1f992e713d1c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:51:25.747Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8ceb79ee-10f0-4986-a54a-1f992e713d1c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:51:28.852Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f457855e-fb0b-4f59-b3e7-c508ffb3ca19 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:51:28.852Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f457855e-fb0b-4f59-b3e7-c508ffb3ca19 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@68b44d33 2021-08-11T16:51:28.852Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f457855e-fb0b-4f59-b3e7-c508ffb3ca19 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:51:28.852Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f457855e-fb0b-4f59-b3e7-c508ffb3ca19 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:51:31.698Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d17ddcd9-8a8a-4851-9a59-b5bdda31143a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:51:31.699Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d17ddcd9-8a8a-4851-9a59-b5bdda31143a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@370a892b 2021-08-11T16:51:31.699Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d17ddcd9-8a8a-4851-9a59-b5bdda31143a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:51:31.699Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d17ddcd9-8a8a-4851-9a59-b5bdda31143a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:51:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:51:33.310Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:51:34.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bba81a9d-3ce2-4a87-af69-0403b6a30ed0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:51:34.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bba81a9d-3ce2-4a87-af69-0403b6a30ed0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1272cd07 2021-08-11T16:51:34.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bba81a9d-3ce2-4a87-af69-0403b6a30ed0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:51:34.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bba81a9d-3ce2-4a87-af69-0403b6a30ed0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:51:37.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6312e8ff-7085-4e99-ad1a-a6a8980df3e2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:51:37.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6312e8ff-7085-4e99-ad1a-a6a8980df3e2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1aeaa5e4 2021-08-11T16:51:37.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6312e8ff-7085-4e99-ad1a-a6a8980df3e2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:51:37.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6312e8ff-7085-4e99-ad1a-a6a8980df3e2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:51:40.953Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa56ab8d-2307-42f8-a785-17563504be57 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:51:40.953Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa56ab8d-2307-42f8-a785-17563504be57 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@507dbe5f 2021-08-11T16:51:40.953Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa56ab8d-2307-42f8-a785-17563504be57 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:51:40.954Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa56ab8d-2307-42f8-a785-17563504be57 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:51:43.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8fd04a6c-ba30-4d22-b84f-ddba4eb4ac20 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:51:43.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8fd04a6c-ba30-4d22-b84f-ddba4eb4ac20 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@335d36aa 2021-08-11T16:51:43.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8fd04a6c-ba30-4d22-b84f-ddba4eb4ac20 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:51:43.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8fd04a6c-ba30-4d22-b84f-ddba4eb4ac20 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:51:46.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f60e8dab-e4b4-4a67-83e3-8ed8910a24f0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:51:46.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f60e8dab-e4b4-4a67-83e3-8ed8910a24f0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3743cf33 2021-08-11T16:51:46.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f60e8dab-e4b4-4a67-83e3-8ed8910a24f0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:51:46.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f60e8dab-e4b4-4a67-83e3-8ed8910a24f0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:51:48.310Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:51:48.317Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:51:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:51:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:51:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:51:49.967Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86d37973-b7e3-442b-ad4e-8a75a0046181 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:51:49.967Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86d37973-b7e3-442b-ad4e-8a75a0046181 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@784999ba 2021-08-11T16:51:49.967Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86d37973-b7e3-442b-ad4e-8a75a0046181 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:51:49.967Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=86d37973-b7e3-442b-ad4e-8a75a0046181 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:51:53.265Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a959dfb-2012-4420-a531-1902a12a9510 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:51:53.265Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a959dfb-2012-4420-a531-1902a12a9510 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@45fb5faa 2021-08-11T16:51:53.265Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a959dfb-2012-4420-a531-1902a12a9510 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:51:53.265Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a959dfb-2012-4420-a531-1902a12a9510 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:51:56.478Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=333ef6bf-bd49-429b-8892-db52fb8f1314 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:51:56.479Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=333ef6bf-bd49-429b-8892-db52fb8f1314 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1f49d9d9 2021-08-11T16:51:56.479Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=333ef6bf-bd49-429b-8892-db52fb8f1314 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:51:56.479Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=333ef6bf-bd49-429b-8892-db52fb8f1314 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:51:59.373Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb915680-f542-448f-8ceb-6ab9e38db29e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:51:59.373Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb915680-f542-448f-8ceb-6ab9e38db29e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@36dab754 2021-08-11T16:51:59.373Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb915680-f542-448f-8ceb-6ab9e38db29e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:51:59.373Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb915680-f542-448f-8ceb-6ab9e38db29e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:52:01.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50cc9a4b-4a7b-4a81-8eac-3c55c440eca6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:52:01.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50cc9a4b-4a7b-4a81-8eac-3c55c440eca6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@26a6c7c 2021-08-11T16:52:01.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50cc9a4b-4a7b-4a81-8eac-3c55c440eca6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:52:01.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50cc9a4b-4a7b-4a81-8eac-3c55c440eca6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:52:03.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:52:03.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:52:04.255Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:52:04.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4302c90c-1cc5-42c1-8ae3-5dbf35e28177 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:52:04.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4302c90c-1cc5-42c1-8ae3-5dbf35e28177 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1df37594 2021-08-11T16:52:04.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4302c90c-1cc5-42c1-8ae3-5dbf35e28177 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:52:04.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4302c90c-1cc5-42c1-8ae3-5dbf35e28177 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:52:07.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d01d01e0-cbba-4763-a1b2-87c7c039d8de PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:52:07.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d01d01e0-cbba-4763-a1b2-87c7c039d8de PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4c25e0e2 2021-08-11T16:52:07.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d01d01e0-cbba-4763-a1b2-87c7c039d8de PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:52:07.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d01d01e0-cbba-4763-a1b2-87c7c039d8de PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:52:10.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9669def-19da-4950-8a54-66c385e13dcd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:52:10.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9669def-19da-4950-8a54-66c385e13dcd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2f269e25 2021-08-11T16:52:10.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9669def-19da-4950-8a54-66c385e13dcd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:52:10.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9669def-19da-4950-8a54-66c385e13dcd PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:52:13.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d06d3609-1a06-49a4-87e7-02e18628735e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:52:13.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d06d3609-1a06-49a4-87e7-02e18628735e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7efe1e31 2021-08-11T16:52:13.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d06d3609-1a06-49a4-87e7-02e18628735e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:52:13.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d06d3609-1a06-49a4-87e7-02e18628735e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:52:16.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e41a0915-f35a-4729-9b7d-4e3a41f94de5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:52:16.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e41a0915-f35a-4729-9b7d-4e3a41f94de5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@78278baf 2021-08-11T16:52:16.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e41a0915-f35a-4729-9b7d-4e3a41f94de5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:52:16.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e41a0915-f35a-4729-9b7d-4e3a41f94de5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:52:18.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:52:18.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:52:19.691Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a1105e4a-3254-4578-a50a-c3bc6679fbbd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:52:19.692Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a1105e4a-3254-4578-a50a-c3bc6679fbbd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1b6656b9 2021-08-11T16:52:19.692Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a1105e4a-3254-4578-a50a-c3bc6679fbbd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:52:19.692Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a1105e4a-3254-4578-a50a-c3bc6679fbbd PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:52:22.632Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3da29c09-cfc6-419e-872d-779c1758ba2a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:52:22.632Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3da29c09-cfc6-419e-872d-779c1758ba2a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@aa3446 2021-08-11T16:52:22.632Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3da29c09-cfc6-419e-872d-779c1758ba2a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:52:22.633Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3da29c09-cfc6-419e-872d-779c1758ba2a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:52:25.690Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a24c47d-98fe-46d0-aaf7-f94b37188512 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:52:25.691Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a24c47d-98fe-46d0-aaf7-f94b37188512 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@eabadf4 2021-08-11T16:52:25.691Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a24c47d-98fe-46d0-aaf7-f94b37188512 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:52:25.691Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a24c47d-98fe-46d0-aaf7-f94b37188512 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:52:28.850Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df48f82f-2e13-4b78-a040-060ca3b8167f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:52:28.851Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df48f82f-2e13-4b78-a040-060ca3b8167f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4a95257d 2021-08-11T16:52:28.851Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df48f82f-2e13-4b78-a040-060ca3b8167f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:52:28.851Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df48f82f-2e13-4b78-a040-060ca3b8167f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:52:31.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c47a75c4-a465-4115-9178-ae3d498dfbfd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:52:31.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c47a75c4-a465-4115-9178-ae3d498dfbfd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@636f6108 2021-08-11T16:52:31.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c47a75c4-a465-4115-9178-ae3d498dfbfd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:52:31.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c47a75c4-a465-4115-9178-ae3d498dfbfd PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:52:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:52:33.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:52:34.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77b7fb65-084a-41a7-9db0-0d86403a5d0f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:52:34.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77b7fb65-084a-41a7-9db0-0d86403a5d0f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@32366127 2021-08-11T16:52:34.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77b7fb65-084a-41a7-9db0-0d86403a5d0f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:52:34.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77b7fb65-084a-41a7-9db0-0d86403a5d0f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:52:37.617Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2adc5169-793a-4ae2-aabd-b3794fb310c1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:52:37.617Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2adc5169-793a-4ae2-aabd-b3794fb310c1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@61a1ae98 2021-08-11T16:52:37.617Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2adc5169-793a-4ae2-aabd-b3794fb310c1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:52:37.617Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2adc5169-793a-4ae2-aabd-b3794fb310c1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:52:40.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=860a056f-58af-46e5-acee-06d46b4f8411 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:52:40.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=860a056f-58af-46e5-acee-06d46b4f8411 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2fcd11c 2021-08-11T16:52:40.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=860a056f-58af-46e5-acee-06d46b4f8411 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:52:40.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=860a056f-58af-46e5-acee-06d46b4f8411 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:52:43.630Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a58b4f0-a410-47ca-9b9d-bd9de6f460a4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:52:43.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a58b4f0-a410-47ca-9b9d-bd9de6f460a4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1a83876a 2021-08-11T16:52:43.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a58b4f0-a410-47ca-9b9d-bd9de6f460a4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:52:43.631Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a58b4f0-a410-47ca-9b9d-bd9de6f460a4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:52:46.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a09ef0c5-7bbc-4435-9ef7-af4ad08873f5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:52:46.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a09ef0c5-7bbc-4435-9ef7-af4ad08873f5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1b3df244 2021-08-11T16:52:46.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a09ef0c5-7bbc-4435-9ef7-af4ad08873f5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:52:46.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a09ef0c5-7bbc-4435-9ef7-af4ad08873f5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:52:48.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:52:48.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:52:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:52:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:52:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:52:50.351Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=61e7bdca-4b97-4a48-97cb-38b6db1c741d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:52:50.352Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=61e7bdca-4b97-4a48-97cb-38b6db1c741d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@77844ad5 2021-08-11T16:52:50.352Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=61e7bdca-4b97-4a48-97cb-38b6db1c741d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:52:50.352Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=61e7bdca-4b97-4a48-97cb-38b6db1c741d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:52:53.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5604357-a097-46f9-b007-9d1797e33a17 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:52:53.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5604357-a097-46f9-b007-9d1797e33a17 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@13940108 2021-08-11T16:52:53.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5604357-a097-46f9-b007-9d1797e33a17 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:52:53.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5604357-a097-46f9-b007-9d1797e33a17 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:52:57.361Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7e11f8e2-5068-475e-ad03-5dacb399b95c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:52:57.361Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7e11f8e2-5068-475e-ad03-5dacb399b95c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@15aa6cc7 2021-08-11T16:52:57.362Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7e11f8e2-5068-475e-ad03-5dacb399b95c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:52:57.362Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7e11f8e2-5068-475e-ad03-5dacb399b95c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T16:52:59.553Z [cluster1-nio-worker-1] WARN c.d.driver.core.HostConnectionPool Error creating connection to cassandra.onap/10.242.184.131:9042 com.datastax.driver.core.exceptions.TransportException: [cassandra.onap/10.242.184.131:9042] Connection has been closed at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1427) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1408) at com.datastax.driver.core.Connection.defunct(Connection.java:585) at com.datastax.driver.core.Connection$Dispatcher.exceptionCaught(Connection.java:1330) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273) at io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.handleReadException(AbstractEpollStreamChannel.java:728) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:818) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-08-11T16:52:59.554Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.184.131:9042 com.datastax.driver.core.exceptions.ConnectionException: [cassandra.onap/10.242.184.131:9042] Pool was closed during initialization at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:173) at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:158) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1080) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.google.common.util.concurrent.CollectionFuture.handleAllCompleted(CollectionFuture.java:69) at com.google.common.util.concurrent.AggregateFuture.processCompleted(AggregateFuture.java:282) at com.google.common.util.concurrent.AggregateFuture.decrementCountAndMaybeComplete(AggregateFuture.java:264) at com.google.common.util.concurrent.AggregateFuture.access$200(AggregateFuture.java:42) at com.google.common.util.concurrent.AggregateFuture$1.run(AggregateFuture.java:147) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:100) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.datastax.driver.core.Connection$Future.onException(Connection.java:1518) at com.datastax.driver.core.Connection$Dispatcher.errorOutAllHandler(Connection.java:1340) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1427) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1408) at com.datastax.driver.core.Connection.defunct(Connection.java:585) at com.datastax.driver.core.Connection$Dispatcher.exceptionCaught(Connection.java:1330) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273) at io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.handleReadException(AbstractEpollStreamChannel.java:728) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:818) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-08-11T16:53:00.175Z [cluster1-nio-worker-1] WARN c.d.driver.core.HostConnectionPool Error creating connection to cassandra.onap/10.242.110.137:9042 com.datastax.driver.core.exceptions.TransportException: [cassandra.onap/10.242.110.137:9042] Connection has been closed at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1427) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1408) at com.datastax.driver.core.Connection.defunct(Connection.java:585) at com.datastax.driver.core.Connection$Dispatcher.exceptionCaught(Connection.java:1330) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273) at io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.handleReadException(AbstractEpollStreamChannel.java:728) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:818) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-08-11T16:53:00.175Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.110.137:9042 com.datastax.driver.core.exceptions.ConnectionException: [cassandra.onap/10.242.110.137:9042] Pool was closed during initialization at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:173) at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:158) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1080) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.google.common.util.concurrent.CollectionFuture.handleAllCompleted(CollectionFuture.java:69) at com.google.common.util.concurrent.AggregateFuture.processCompleted(AggregateFuture.java:282) at com.google.common.util.concurrent.AggregateFuture.decrementCountAndMaybeComplete(AggregateFuture.java:264) at com.google.common.util.concurrent.AggregateFuture.access$200(AggregateFuture.java:42) at com.google.common.util.concurrent.AggregateFuture$1.run(AggregateFuture.java:147) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:100) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.datastax.driver.core.Connection$Future.onException(Connection.java:1518) at com.datastax.driver.core.Connection$Dispatcher.errorOutAllHandler(Connection.java:1340) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1427) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1408) at com.datastax.driver.core.Connection.defunct(Connection.java:585) at com.datastax.driver.core.Connection$Dispatcher.exceptionCaught(Connection.java:1330) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273) at io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.handleReadException(AbstractEpollStreamChannel.java:728) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:818) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-08-11T16:53:00.175Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a44b9b07-8660-4d3e-a839-5b16cef6c752 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:53:00.175Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a44b9b07-8660-4d3e-a839-5b16cef6c752 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@61f103e4 2021-08-11T16:53:00.175Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a44b9b07-8660-4d3e-a839-5b16cef6c752 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:53:00.175Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a44b9b07-8660-4d3e-a839-5b16cef6c752 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:53:00.214Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=a44b9b07-8660-4d3e-a839-5b16cef6c752 ErrorCategory=INFO ServiceName=BE Component CASSANDRA state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component CASSANDRA state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:53:00.214Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=a44b9b07-8660-4d3e-a839-5b16cef6c752 ErrorCategory=ERROR ServiceName=BE Component CASSANDRA state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-08-11T16:53:01.454Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:01.506Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:01.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b13459c6-1a4c-4f7f-ab8f-5cf774263476 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:53:01.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b13459c6-1a4c-4f7f-ab8f-5cf774263476 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@34763fed 2021-08-11T16:53:01.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b13459c6-1a4c-4f7f-ab8f-5cf774263476 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:53:01.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b13459c6-1a4c-4f7f-ab8f-5cf774263476 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:53:01.608Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:01.710Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:01.813Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:01.915Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:02.019Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:02.221Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:02.247Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:02.303Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:02.405Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:02.425Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:02.608Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:02.628Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:02.810Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:03.030Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:03.213Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:03.252Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:53:03.253Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-08-11T16:53:03.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:53:03.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:53:03.833Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:04.016Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:04.258Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:53:04.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e093843-fa7b-4cfd-b7d9-7beef7c815ab PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:53:04.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e093843-fa7b-4cfd-b7d9-7beef7c815ab PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@759e42a 2021-08-11T16:53:04.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e093843-fa7b-4cfd-b7d9-7beef7c815ab PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:53:04.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e093843-fa7b-4cfd-b7d9-7beef7c815ab PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:53:04.595Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=1e093843-fa7b-4cfd-b7d9-7beef7c815ab ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:53:04.596Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=1e093843-fa7b-4cfd-b7d9-7beef7c815ab ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-08-11T16:53:04.635Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:04.819Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:05.438Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T16:52:55.947055Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:05.621Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:06.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:53:06.423Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:07.226Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:07.599Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55db7252-5fd4-40bd-9540-6fe683a29c07 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:53:07.599Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55db7252-5fd4-40bd-9540-6fe683a29c07 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@41536c52 2021-08-11T16:53:07.599Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55db7252-5fd4-40bd-9540-6fe683a29c07 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:53:07.599Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55db7252-5fd4-40bd-9540-6fe683a29c07 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:53:08.028Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:08.830Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:09.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:53:09.648Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:10.446Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:10.450Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:10.498Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:10.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea05e0cc-282b-4439-856d-fc9aec008f8b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:53:10.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea05e0cc-282b-4439-856d-fc9aec008f8b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2854a9bf 2021-08-11T16:53:10.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea05e0cc-282b-4439-856d-fc9aec008f8b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:53:10.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea05e0cc-282b-4439-856d-fc9aec008f8b PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:53:10.601Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:10.804Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:11.007Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:11.410Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:11.813Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:12.055Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:12.083Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:12.135Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:12.216Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:12.237Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:12.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:53:12.340Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:12.543Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:12.618Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:12.746Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:12.949Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:13.152Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:13.421Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:13.554Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:13.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eea4b6fa-64dc-4cfa-8e4d-a8e0b68db6ec PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:53:13.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eea4b6fa-64dc-4cfa-8e4d-a8e0b68db6ec PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2d58620f 2021-08-11T16:53:13.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eea4b6fa-64dc-4cfa-8e4d-a8e0b68db6ec PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:53:13.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eea4b6fa-64dc-4cfa-8e4d-a8e0b68db6ec PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:53:14.224Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T16:52:55.947055Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:14.356Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:15.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:53:15.959Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:16.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=51b34e67-c5a9-4393-9b57-23abe9be4630 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:53:16.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=51b34e67-c5a9-4393-9b57-23abe9be4630 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6568ab3a 2021-08-11T16:53:16.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=51b34e67-c5a9-4393-9b57-23abe9be4630 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:53:16.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=51b34e67-c5a9-4393-9b57-23abe9be4630 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:53:18.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:53:18.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:53:18.318Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:53:19.164Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:19.191Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:19.227Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:19.244Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:19.279Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:19.346Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:19.382Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:19.485Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:19.549Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:19.595Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=de31b38b-fc8e-438d-aa95-9373e60e2bd5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:53:19.596Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=de31b38b-fc8e-438d-aa95-9373e60e2bd5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3f836a39 2021-08-11T16:53:19.596Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=de31b38b-fc8e-438d-aa95-9373e60e2bd5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:53:19.596Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=de31b38b-fc8e-438d-aa95-9373e60e2bd5 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:53:19.687Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:19.891Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:19.951Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:20.294Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:20.354Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:20.757Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:21.097Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:21.159Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:21.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:53:21.962Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:22.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ceed8d2d-7abc-4a59-b6aa-29f3d682c855 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:53:22.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ceed8d2d-7abc-4a59-b6aa-29f3d682c855 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@32973527 2021-08-11T16:53:22.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ceed8d2d-7abc-4a59-b6aa-29f3d682c855 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:53:22.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ceed8d2d-7abc-4a59-b6aa-29f3d682c855 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:53:22.700Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T16:52:55.947055Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:22.765Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:24.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:53:24.367Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:25.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0315d0bc-5df4-42f2-a779-e6787eff513e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:53:25.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0315d0bc-5df4-42f2-a779-e6787eff513e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5c317160 2021-08-11T16:53:25.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0315d0bc-5df4-42f2-a779-e6787eff513e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:53:25.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0315d0bc-5df4-42f2-a779-e6787eff513e PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:53:26.047Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:26.147Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:26.250Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:26.452Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:26.855Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:27.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:53:27.658Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:27.703Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:27.755Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:27.858Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:28.061Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:28.463Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:308) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:422) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:28.464Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:28.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=acff3ea6-4e2b-477a-8310-32e63dcc9a45 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:53:28.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=acff3ea6-4e2b-477a-8310-32e63dcc9a45 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7c2030a7 2021-08-11T16:53:28.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=acff3ea6-4e2b-477a-8310-32e63dcc9a45 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:53:28.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=acff3ea6-4e2b-477a-8310-32e63dcc9a45 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:53:29.266Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:308) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:422) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:29.266Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:30.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:53:30.868Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:30.869Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T16:52:55.947055Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:31.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3777aebc-28d9-4c7e-af9a-e2bcd136fe9e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:53:31.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3777aebc-28d9-4c7e-af9a-e2bcd136fe9e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@58564fb8 2021-08-11T16:53:31.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3777aebc-28d9-4c7e-af9a-e2bcd136fe9e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:53:31.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3777aebc-28d9-4c7e-af9a-e2bcd136fe9e PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:53:32.471Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:33.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:53:33.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:53:33.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:53:34.576Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=54775d61-beeb-4a68-bf5d-65feca6fcc1f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:53:34.576Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=54775d61-beeb-4a68-bf5d-65feca6fcc1f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@53636798 2021-08-11T16:53:34.576Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=54775d61-beeb-4a68-bf5d-65feca6fcc1f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:53:34.576Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=54775d61-beeb-4a68-bf5d-65feca6fcc1f PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:53:35.677Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:35.730Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:35.782Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:35.871Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:35.884Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:35.898Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:35.926Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:35.958Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:35.999Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:35.999Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:308) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:422) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:36.027Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:36.054Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:36.102Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:36.106Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:36.158Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:36.204Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:36.210Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:36.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:53:36.307Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:36.312Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:36.409Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:36.414Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:36.611Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:36.616Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:36.818Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:37.013Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:37.220Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:37.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6560ff02-3253-4d89-9577-87732de6c812 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:53:37.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6560ff02-3253-4d89-9577-87732de6c812 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@40eb6e5 2021-08-11T16:53:37.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6560ff02-3253-4d89-9577-87732de6c812 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:53:37.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6560ff02-3253-4d89-9577-87732de6c812 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:53:37.816Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:38.048Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:38.619Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:38.850Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:39.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:53:39.654Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T16:52:55.947055Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:40.248Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:40.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dce3fec9-f744-4407-a562-415e05496310 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:53:40.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dce3fec9-f744-4407-a562-415e05496310 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@79d61bcb 2021-08-11T16:53:40.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dce3fec9-f744-4407-a562-415e05496310 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:53:40.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dce3fec9-f744-4407-a562-415e05496310 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:53:42.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:53:43.452Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:43.479Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:43.506Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:43.548Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:43.600Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:43.604Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2badc403-af41-45e5-94fc-0f4dec3ba561 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:53:43.604Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2badc403-af41-45e5-94fc-0f4dec3ba561 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1397b0c6 2021-08-11T16:53:43.604Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2badc403-af41-45e5-94fc-0f4dec3ba561 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:53:43.604Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2badc403-af41-45e5-94fc-0f4dec3ba561 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:53:43.651Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:43.703Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:43.755Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:43.807Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:43.910Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:44.121Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:44.324Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:44.526Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:44.656Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:44.709Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:44.728Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:44.815Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T16:53:45.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T16:53:46.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb8153de-360b-48ab-8785-f5c6d592656d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:53:46.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb8153de-360b-48ab-8785-f5c6d592656d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5b004a8b 2021-08-11T16:53:46.590Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb8153de-360b-48ab-8785-f5c6d592656d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:53:46.590Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb8153de-360b-48ab-8785-f5c6d592656d PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:53:47.260Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-08-11T16:53:48.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:53:48.317Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:53:49.290Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:53:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:53:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:53:49.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a084cfa6-3f53-4ec8-856e-c092f2e9a816 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:53:49.576Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a084cfa6-3f53-4ec8-856e-c092f2e9a816 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2ac87690 2021-08-11T16:53:49.576Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a084cfa6-3f53-4ec8-856e-c092f2e9a816 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:53:49.576Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a084cfa6-3f53-4ec8-856e-c092f2e9a816 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:53:50.058Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=a084cfa6-3f53-4ec8-856e-c092f2e9a816 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:53:50.059Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=a084cfa6-3f53-4ec8-856e-c092f2e9a816 ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-08-11T16:53:52.576Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eadd4eb9-6ccf-48a9-86a0-3bc49a86383b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:53:52.577Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eadd4eb9-6ccf-48a9-86a0-3bc49a86383b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@63092841 2021-08-11T16:53:52.577Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eadd4eb9-6ccf-48a9-86a0-3bc49a86383b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:53:52.577Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eadd4eb9-6ccf-48a9-86a0-3bc49a86383b PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:53:55.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1f97917c-0006-4ae9-866e-c7db72b6ce60 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:53:55.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1f97917c-0006-4ae9-866e-c7db72b6ce60 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4db6b566 2021-08-11T16:53:55.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1f97917c-0006-4ae9-866e-c7db72b6ce60 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:53:55.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1f97917c-0006-4ae9-866e-c7db72b6ce60 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T16:54:01.054Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=976b7dbc-5b6f-4f82-9994-7e0cec03d99c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:54:01.055Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=976b7dbc-5b6f-4f82-9994-7e0cec03d99c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@143bea49 2021-08-11T16:54:01.055Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=976b7dbc-5b6f-4f82-9994-7e0cec03d99c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:54:01.055Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=976b7dbc-5b6f-4f82-9994-7e0cec03d99c PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:54:03.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:54:03.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:54:04.061Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=MAJOR RequestID=976b7dbc-5b6f-4f82-9994-7e0cec03d99c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=ExecuteRestRequest Internal flow error. Operation: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out 2021-08-11T16:54:04.063Z [BE-Health-Check-Task] ERROR o.o.s.b.c.h.HealthCheckBusinessLogic RequestID=976b7dbc-5b6f-4f82-9994-7e0cec03d99c ErrorCategory=ERROR ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=500 PartnerName=SDC.BE TargetEntity=null ON_BOARDING unexpected response: org.openecomp.sdc.common.http.client.api.HttpExecuteException: Execute request GET https://sdc-onboarding-be.onap:8445/onboarding-api/v1.0/healthcheck HTTP/1.1 failed with exception: Read timed out at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:124) at org.openecomp.sdc.common.http.client.api.HttpClient.get(HttpClient.java:66) at org.openecomp.sdc.common.http.client.api.HttpRequestHandler.get(HttpRequestHandler.java:78) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:51) at org.openecomp.sdc.common.http.client.api.HttpRequest.get(HttpRequest.java:47) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getHostedComponentsBeHealthCheck(HealthCheckBusinessLogic.java:232) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic.getBeHealthCheckInfos(HealthCheckBusinessLogic.java:145) at org.openecomp.sdc.be.components.health.HealthCheckBusinessLogic$HealthCheckScheduledTask.run(HealthCheckBusinessLogic.java:405) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.net.SocketTimeoutException: Read timed out at java.base/java.net.SocketInputStream.socketRead0(Native Method) at java.base/java.net.SocketInputStream.socketRead(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/java.net.SocketInputStream.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.read(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(Unknown Source) at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(Unknown Source) at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(Unknown Source) at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:139) at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:155) at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:284) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:261) at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:165) at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165) at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:272) at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:124) at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111) at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.openecomp.sdc.common.http.client.api.HttpClient.execute(HttpClient.java:118) ... 13 common frames omitted 2021-08-11T16:54:04.065Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=976b7dbc-5b6f-4f82-9994-7e0cec03d99c ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component CASSANDRA state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:54:04.065Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=976b7dbc-5b6f-4f82-9994-7e0cec03d99c ErrorCategory=INFO ServiceName=ExecuteRestRequest TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:54:04.278Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:54:04.386Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=57880ec9-8427-44b6-8011-8ce14498177f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:54:04.386Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=57880ec9-8427-44b6-8011-8ce14498177f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7dae62a8 2021-08-11T16:54:04.386Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=57880ec9-8427-44b6-8011-8ce14498177f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:54:04.386Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=57880ec9-8427-44b6-8011-8ce14498177f PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:54:05.670Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=57880ec9-8427-44b6-8011-8ce14498177f ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:54:05.670Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=57880ec9-8427-44b6-8011-8ce14498177f ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T16:54:06.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ba807dd-5476-4726-8129-0be422bfd213 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:54:06.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ba807dd-5476-4726-8129-0be422bfd213 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2faa9466 2021-08-11T16:54:06.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ba807dd-5476-4726-8129-0be422bfd213 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:54:06.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ba807dd-5476-4726-8129-0be422bfd213 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:54:07.694Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e7976236-3725-427b-9505-85a724633989 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:54:07.694Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e7976236-3725-427b-9505-85a724633989 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@64084a78 2021-08-11T16:54:07.695Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e7976236-3725-427b-9505-85a724633989 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:54:07.695Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e7976236-3725-427b-9505-85a724633989 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:54:11.351Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ae8c1d7-f393-4224-92b0-6ce689980168 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:54:11.351Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ae8c1d7-f393-4224-92b0-6ce689980168 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6a3ddf73 2021-08-11T16:54:11.351Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ae8c1d7-f393-4224-92b0-6ce689980168 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:54:11.351Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2ae8c1d7-f393-4224-92b0-6ce689980168 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:54:13.617Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c2aaca35-57f4-4b91-bd3f-01e1142a8e00 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:54:13.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c2aaca35-57f4-4b91-bd3f-01e1142a8e00 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@45ee48be 2021-08-11T16:54:13.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c2aaca35-57f4-4b91-bd3f-01e1142a8e00 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:54:13.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c2aaca35-57f4-4b91-bd3f-01e1142a8e00 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:54:16.613Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1dfde68b-a19b-496c-9798-804c5315f2a0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:54:16.613Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1dfde68b-a19b-496c-9798-804c5315f2a0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3e1cedc1 2021-08-11T16:54:16.613Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1dfde68b-a19b-496c-9798-804c5315f2a0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:54:16.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1dfde68b-a19b-496c-9798-804c5315f2a0 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:54:18.305Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:54:18.311Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:54:19.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=57331182-87fa-4062-8b3c-1876ee7a1c8f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:54:19.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=57331182-87fa-4062-8b3c-1876ee7a1c8f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@18612bf0 2021-08-11T16:54:19.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=57331182-87fa-4062-8b3c-1876ee7a1c8f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:54:19.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=57331182-87fa-4062-8b3c-1876ee7a1c8f PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:54:22.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e79cb61-db2d-462b-b96c-0d031d560ba0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:54:22.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e79cb61-db2d-462b-b96c-0d031d560ba0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1ca3b9c8 2021-08-11T16:54:22.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e79cb61-db2d-462b-b96c-0d031d560ba0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:54:22.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9e79cb61-db2d-462b-b96c-0d031d560ba0 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:54:25.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4d0febff-0c32-4212-be58-7186611f0aae PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:54:25.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4d0febff-0c32-4212-be58-7186611f0aae PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2b3fb95a 2021-08-11T16:54:25.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4d0febff-0c32-4212-be58-7186611f0aae PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:54:25.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4d0febff-0c32-4212-be58-7186611f0aae PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:54:28.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c2da817a-0b57-420d-ba20-39ded5b668ce PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:54:28.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c2da817a-0b57-420d-ba20-39ded5b668ce PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@72143b19 2021-08-11T16:54:28.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c2da817a-0b57-420d-ba20-39ded5b668ce PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:54:28.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c2da817a-0b57-420d-ba20-39ded5b668ce PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:54:31.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e02f7199-ad8d-4b0a-936b-2eb1bdfb7e05 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:54:31.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e02f7199-ad8d-4b0a-936b-2eb1bdfb7e05 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@63bbb000 2021-08-11T16:54:31.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e02f7199-ad8d-4b0a-936b-2eb1bdfb7e05 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:54:31.955Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e02f7199-ad8d-4b0a-936b-2eb1bdfb7e05 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:54:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:54:33.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:54:34.771Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc08afce-4984-42cf-9c9f-3ef2564c9cd4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:54:34.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc08afce-4984-42cf-9c9f-3ef2564c9cd4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@62572846 2021-08-11T16:54:34.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc08afce-4984-42cf-9c9f-3ef2564c9cd4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:54:34.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc08afce-4984-42cf-9c9f-3ef2564c9cd4 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:54:37.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6cd6627-63d6-47af-9855-aa6f24dc7231 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:54:37.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6cd6627-63d6-47af-9855-aa6f24dc7231 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@34f4a83f 2021-08-11T16:54:37.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6cd6627-63d6-47af-9855-aa6f24dc7231 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:54:37.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e6cd6627-63d6-47af-9855-aa6f24dc7231 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:54:40.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d89e2312-39f1-410e-9ba8-47a36ab3aec3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:54:40.611Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d89e2312-39f1-410e-9ba8-47a36ab3aec3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@676b814b 2021-08-11T16:54:40.611Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d89e2312-39f1-410e-9ba8-47a36ab3aec3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:54:40.611Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d89e2312-39f1-410e-9ba8-47a36ab3aec3 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:54:43.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41ab80d8-14f2-4995-9a23-781cb95a020f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:54:43.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41ab80d8-14f2-4995-9a23-781cb95a020f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@59ee1616 2021-08-11T16:54:43.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41ab80d8-14f2-4995-9a23-781cb95a020f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:54:43.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41ab80d8-14f2-4995-9a23-781cb95a020f PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:54:46.609Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0afc7cd9-edc4-490e-82cb-29abb2f2ceba PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:54:46.609Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0afc7cd9-edc4-490e-82cb-29abb2f2ceba PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2a01075 2021-08-11T16:54:46.609Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0afc7cd9-edc4-490e-82cb-29abb2f2ceba PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:54:46.609Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0afc7cd9-edc4-490e-82cb-29abb2f2ceba PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:54:48.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:54:48.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:54:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:54:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:54:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:54:49.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c3fa68b-f246-4fac-a3bb-f0466ecae1c2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:54:49.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c3fa68b-f246-4fac-a3bb-f0466ecae1c2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1304f54e 2021-08-11T16:54:49.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c3fa68b-f246-4fac-a3bb-f0466ecae1c2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:54:49.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c3fa68b-f246-4fac-a3bb-f0466ecae1c2 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:54:52.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d6db8c71-5baf-4e69-99a8-74eba794d078 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:54:52.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d6db8c71-5baf-4e69-99a8-74eba794d078 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@28de4fbc 2021-08-11T16:54:52.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d6db8c71-5baf-4e69-99a8-74eba794d078 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:54:52.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d6db8c71-5baf-4e69-99a8-74eba794d078 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:54:55.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c54f16d5-f8ae-4ef6-980c-511ba0506560 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:54:55.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c54f16d5-f8ae-4ef6-980c-511ba0506560 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1a9a217 2021-08-11T16:54:55.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c54f16d5-f8ae-4ef6-980c-511ba0506560 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:54:55.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c54f16d5-f8ae-4ef6-980c-511ba0506560 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:54:58.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba73d322-4261-4883-a509-1d861dabb38c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:54:58.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba73d322-4261-4883-a509-1d861dabb38c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@38829ef 2021-08-11T16:54:58.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba73d322-4261-4883-a509-1d861dabb38c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:54:58.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba73d322-4261-4883-a509-1d861dabb38c PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:55:01.606Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=596a735b-d56d-480b-ae28-f54695131ef9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:55:01.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=596a735b-d56d-480b-ae28-f54695131ef9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@37121255 2021-08-11T16:55:01.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=596a735b-d56d-480b-ae28-f54695131ef9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:55:01.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=596a735b-d56d-480b-ae28-f54695131ef9 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:55:03.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:55:03.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:55:04.288Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:55:04.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=202c1642-34aa-4123-9684-03ee390522a3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:55:04.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=202c1642-34aa-4123-9684-03ee390522a3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6deb08a4 2021-08-11T16:55:04.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=202c1642-34aa-4123-9684-03ee390522a3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:55:04.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=202c1642-34aa-4123-9684-03ee390522a3 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:55:07.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d8145e4-5d4d-4b2e-840b-0fc9483a9d47 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:55:07.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d8145e4-5d4d-4b2e-840b-0fc9483a9d47 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@219eb614 2021-08-11T16:55:07.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d8145e4-5d4d-4b2e-840b-0fc9483a9d47 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:55:07.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d8145e4-5d4d-4b2e-840b-0fc9483a9d47 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:55:10.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=65f4faec-9c80-4d14-93a6-3183a92a981d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:55:10.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=65f4faec-9c80-4d14-93a6-3183a92a981d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@639b2499 2021-08-11T16:55:10.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=65f4faec-9c80-4d14-93a6-3183a92a981d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:55:10.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=65f4faec-9c80-4d14-93a6-3183a92a981d PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:55:13.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c9e61fc2-4171-4ba5-bb15-47ceaac8b335 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:55:13.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c9e61fc2-4171-4ba5-bb15-47ceaac8b335 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@443c0205 2021-08-11T16:55:13.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c9e61fc2-4171-4ba5-bb15-47ceaac8b335 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:55:13.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c9e61fc2-4171-4ba5-bb15-47ceaac8b335 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:55:16.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24ad2e5a-b835-4e89-a287-ab82e1f7a85c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:55:16.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24ad2e5a-b835-4e89-a287-ab82e1f7a85c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@71279c0e 2021-08-11T16:55:16.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24ad2e5a-b835-4e89-a287-ab82e1f7a85c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:55:16.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24ad2e5a-b835-4e89-a287-ab82e1f7a85c PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:55:18.309Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:55:18.317Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:55:19.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3081ddb-e35e-4868-9c24-34da3dcbdbae PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:55:19.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3081ddb-e35e-4868-9c24-34da3dcbdbae PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@25ddaa28 2021-08-11T16:55:19.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3081ddb-e35e-4868-9c24-34da3dcbdbae PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:55:19.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3081ddb-e35e-4868-9c24-34da3dcbdbae PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:55:22.654Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f84ecd67-fe93-49a8-a8a9-4d02604c622f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:55:22.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f84ecd67-fe93-49a8-a8a9-4d02604c622f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1f58ede6 2021-08-11T16:55:22.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f84ecd67-fe93-49a8-a8a9-4d02604c622f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:55:22.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f84ecd67-fe93-49a8-a8a9-4d02604c622f PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:55:25.690Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7270598e-d0bc-4e42-873e-71e2f4939866 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:55:25.691Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7270598e-d0bc-4e42-873e-71e2f4939866 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3b50f822 2021-08-11T16:55:25.691Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7270598e-d0bc-4e42-873e-71e2f4939866 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:55:25.691Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7270598e-d0bc-4e42-873e-71e2f4939866 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:55:28.952Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c07df12-31bf-494d-a224-8cd6371fd0bf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:55:28.952Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c07df12-31bf-494d-a224-8cd6371fd0bf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@76dac4aa 2021-08-11T16:55:28.952Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c07df12-31bf-494d-a224-8cd6371fd0bf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:55:28.952Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3c07df12-31bf-494d-a224-8cd6371fd0bf PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:55:31.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=de977d8c-c398-440d-b86d-c6317930ebfa PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:55:31.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=de977d8c-c398-440d-b86d-c6317930ebfa PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3e1ebcb3 2021-08-11T16:55:31.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=de977d8c-c398-440d-b86d-c6317930ebfa PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:55:31.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=de977d8c-c398-440d-b86d-c6317930ebfa PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:55:33.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:55:33.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:55:34.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2610365-141e-42df-a055-259dbb05170c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:55:34.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2610365-141e-42df-a055-259dbb05170c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6671414b 2021-08-11T16:55:34.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2610365-141e-42df-a055-259dbb05170c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:55:34.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2610365-141e-42df-a055-259dbb05170c PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:55:37.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe1f4465-2c0d-4105-b4b8-84e97d28cf40 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:55:37.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe1f4465-2c0d-4105-b4b8-84e97d28cf40 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5adc244f 2021-08-11T16:55:37.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe1f4465-2c0d-4105-b4b8-84e97d28cf40 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:55:37.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe1f4465-2c0d-4105-b4b8-84e97d28cf40 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:55:40.854Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b879a0a-8a11-44a6-a83c-9c49f0b556e5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:55:40.854Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b879a0a-8a11-44a6-a83c-9c49f0b556e5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7a03542a 2021-08-11T16:55:40.854Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b879a0a-8a11-44a6-a83c-9c49f0b556e5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:55:40.854Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b879a0a-8a11-44a6-a83c-9c49f0b556e5 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:55:43.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8305ad59-d369-47fc-a5ef-ae5b20c055b6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:55:43.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8305ad59-d369-47fc-a5ef-ae5b20c055b6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@31b394fd 2021-08-11T16:55:43.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8305ad59-d369-47fc-a5ef-ae5b20c055b6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:55:43.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8305ad59-d369-47fc-a5ef-ae5b20c055b6 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:55:46.708Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae2d3b83-e7f3-4c05-b660-91ba964a73b1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:55:46.708Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae2d3b83-e7f3-4c05-b660-91ba964a73b1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5da4e099 2021-08-11T16:55:46.708Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae2d3b83-e7f3-4c05-b660-91ba964a73b1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:55:46.708Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ae2d3b83-e7f3-4c05-b660-91ba964a73b1 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:55:48.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:55:48.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:55:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:55:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:55:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:55:50.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8576087b-4809-4851-a57c-7ef76f3c0b5c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:55:50.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8576087b-4809-4851-a57c-7ef76f3c0b5c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6661593e 2021-08-11T16:55:50.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8576087b-4809-4851-a57c-7ef76f3c0b5c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:55:50.070Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8576087b-4809-4851-a57c-7ef76f3c0b5c PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:55:53.345Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e201f22-df5b-4cc5-8243-3e7a53b1d58f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:55:53.345Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e201f22-df5b-4cc5-8243-3e7a53b1d58f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@623e3dc 2021-08-11T16:55:53.345Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e201f22-df5b-4cc5-8243-3e7a53b1d58f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:55:53.345Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e201f22-df5b-4cc5-8243-3e7a53b1d58f PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:55:56.048Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=931600ae-90bd-466c-8430-516df24502bf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:55:56.048Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=931600ae-90bd-466c-8430-516df24502bf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2b1a8b99 2021-08-11T16:55:56.048Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=931600ae-90bd-466c-8430-516df24502bf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:55:56.048Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=931600ae-90bd-466c-8430-516df24502bf PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:55:58.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6663124f-9753-4895-9eb5-260f93e87fbc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:55:58.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6663124f-9753-4895-9eb5-260f93e87fbc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5140064b 2021-08-11T16:55:58.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6663124f-9753-4895-9eb5-260f93e87fbc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:55:58.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6663124f-9753-4895-9eb5-260f93e87fbc PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:56:01.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=91cd17ad-0f9d-4ab4-96f3-0848aed73720 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:56:01.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=91cd17ad-0f9d-4ab4-96f3-0848aed73720 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3d041df4 2021-08-11T16:56:01.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=91cd17ad-0f9d-4ab4-96f3-0848aed73720 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:56:01.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=91cd17ad-0f9d-4ab4-96f3-0848aed73720 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:56:03.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:56:03.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:56:04.281Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:56:04.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a0deccc-91d9-4008-97d1-f7b7c303f748 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:56:04.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a0deccc-91d9-4008-97d1-f7b7c303f748 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2420248b 2021-08-11T16:56:04.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a0deccc-91d9-4008-97d1-f7b7c303f748 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:56:04.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5a0deccc-91d9-4008-97d1-f7b7c303f748 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:56:07.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6072a57c-8345-44be-8d72-5622b0e623be PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:56:07.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6072a57c-8345-44be-8d72-5622b0e623be PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5b9925d5 2021-08-11T16:56:07.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6072a57c-8345-44be-8d72-5622b0e623be PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:56:07.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6072a57c-8345-44be-8d72-5622b0e623be PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:56:10.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9a81464-bdd6-470d-9379-16e02bbb7c03 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:56:10.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9a81464-bdd6-470d-9379-16e02bbb7c03 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@a3a12b3 2021-08-11T16:56:10.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9a81464-bdd6-470d-9379-16e02bbb7c03 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:56:10.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9a81464-bdd6-470d-9379-16e02bbb7c03 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:56:13.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc3663a0-5ba1-44a0-9b58-a6d603007922 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:56:13.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc3663a0-5ba1-44a0-9b58-a6d603007922 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@17cc45bd 2021-08-11T16:56:13.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc3663a0-5ba1-44a0-9b58-a6d603007922 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:56:13.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc3663a0-5ba1-44a0-9b58-a6d603007922 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:56:16.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd241ec2-ddf1-4937-b1da-5e796f724836 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:56:16.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd241ec2-ddf1-4937-b1da-5e796f724836 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@32ed3fb3 2021-08-11T16:56:16.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd241ec2-ddf1-4937-b1da-5e796f724836 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:56:16.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd241ec2-ddf1-4937-b1da-5e796f724836 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:56:18.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:56:18.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:56:19.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac9a31b5-5a45-42d9-bffb-db41243bfb00 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:56:19.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac9a31b5-5a45-42d9-bffb-db41243bfb00 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@204803b6 2021-08-11T16:56:19.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac9a31b5-5a45-42d9-bffb-db41243bfb00 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:56:19.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac9a31b5-5a45-42d9-bffb-db41243bfb00 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:56:22.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2586eac4-80cf-4bf4-8ba3-1bb4b45e7afa PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:56:22.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2586eac4-80cf-4bf4-8ba3-1bb4b45e7afa PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7a3d2286 2021-08-11T16:56:22.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2586eac4-80cf-4bf4-8ba3-1bb4b45e7afa PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:56:22.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2586eac4-80cf-4bf4-8ba3-1bb4b45e7afa PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:56:25.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3a7a683e-d338-4134-8b99-5ab777d27bfa PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:56:25.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3a7a683e-d338-4134-8b99-5ab777d27bfa PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@fa971d1 2021-08-11T16:56:25.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3a7a683e-d338-4134-8b99-5ab777d27bfa PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:56:25.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3a7a683e-d338-4134-8b99-5ab777d27bfa PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:56:28.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70cdcb9d-2cc0-4ba9-ad59-1766deefd7bc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:56:28.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70cdcb9d-2cc0-4ba9-ad59-1766deefd7bc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@54e2d730 2021-08-11T16:56:28.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70cdcb9d-2cc0-4ba9-ad59-1766deefd7bc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:56:28.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70cdcb9d-2cc0-4ba9-ad59-1766deefd7bc PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:56:31.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=594ccda3-020b-473a-b685-364508756757 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:56:31.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=594ccda3-020b-473a-b685-364508756757 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@34d72ba9 2021-08-11T16:56:31.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=594ccda3-020b-473a-b685-364508756757 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:56:31.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=594ccda3-020b-473a-b685-364508756757 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:56:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:56:33.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:56:34.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=84d396f2-1829-43b3-876e-463108d5a65d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:56:34.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=84d396f2-1829-43b3-876e-463108d5a65d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@27ed06a7 2021-08-11T16:56:34.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=84d396f2-1829-43b3-876e-463108d5a65d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:56:34.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=84d396f2-1829-43b3-876e-463108d5a65d PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:56:37.654Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5bac53c-a128-4ef2-9261-e780c7a79500 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:56:37.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5bac53c-a128-4ef2-9261-e780c7a79500 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@630ecc72 2021-08-11T16:56:37.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5bac53c-a128-4ef2-9261-e780c7a79500 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:56:37.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d5bac53c-a128-4ef2-9261-e780c7a79500 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:56:40.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=43378a10-9321-416a-b579-9e943550076e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:56:40.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=43378a10-9321-416a-b579-9e943550076e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2197cc74 2021-08-11T16:56:40.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=43378a10-9321-416a-b579-9e943550076e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:56:40.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=43378a10-9321-416a-b579-9e943550076e PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:56:43.603Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7572a0c1-60e0-4213-88b5-b4a8abbce654 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:56:43.604Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7572a0c1-60e0-4213-88b5-b4a8abbce654 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2c10318c 2021-08-11T16:56:43.604Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7572a0c1-60e0-4213-88b5-b4a8abbce654 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:56:43.604Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7572a0c1-60e0-4213-88b5-b4a8abbce654 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:56:46.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03306498-1416-4d42-8d80-932399d8df1e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:56:46.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03306498-1416-4d42-8d80-932399d8df1e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6307c297 2021-08-11T16:56:46.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03306498-1416-4d42-8d80-932399d8df1e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:56:46.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=03306498-1416-4d42-8d80-932399d8df1e PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:56:48.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:56:48.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:56:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:56:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:56:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:56:50.255Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb88ea98-aba7-4b93-b1c6-faf3fa3629e7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:56:50.255Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb88ea98-aba7-4b93-b1c6-faf3fa3629e7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@767fffd9 2021-08-11T16:56:50.255Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb88ea98-aba7-4b93-b1c6-faf3fa3629e7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:56:50.256Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb88ea98-aba7-4b93-b1c6-faf3fa3629e7 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:56:53.363Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=395ce62a-9610-42a3-9f20-bc1851f638a9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:56:53.363Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=395ce62a-9610-42a3-9f20-bc1851f638a9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6994690e 2021-08-11T16:56:53.363Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=395ce62a-9610-42a3-9f20-bc1851f638a9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:56:53.364Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=395ce62a-9610-42a3-9f20-bc1851f638a9 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:56:56.456Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=07fc1e9e-ed1c-42d1-99cb-4006a8720d86 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:56:56.456Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=07fc1e9e-ed1c-42d1-99cb-4006a8720d86 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4425a9e1 2021-08-11T16:56:56.456Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=07fc1e9e-ed1c-42d1-99cb-4006a8720d86 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:56:56.456Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=07fc1e9e-ed1c-42d1-99cb-4006a8720d86 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:57:00.148Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=919d149f-2b16-4bee-8c33-820d06252a5c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:57:00.148Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=919d149f-2b16-4bee-8c33-820d06252a5c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@196bc888 2021-08-11T16:57:00.148Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=919d149f-2b16-4bee-8c33-820d06252a5c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:57:00.148Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=919d149f-2b16-4bee-8c33-820d06252a5c PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:57:02.356Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6caa259-a9d4-4946-ba4a-af355aa35247 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:57:02.356Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6caa259-a9d4-4946-ba4a-af355aa35247 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@10637d4b 2021-08-11T16:57:02.356Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6caa259-a9d4-4946-ba4a-af355aa35247 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:57:02.357Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c6caa259-a9d4-4946-ba4a-af355aa35247 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:57:03.316Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:57:03.327Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:57:04.263Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:57:04.687Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d77ddf4c-20f1-4154-8b8c-33006a1ae79b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:57:04.687Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d77ddf4c-20f1-4154-8b8c-33006a1ae79b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5a75aa50 2021-08-11T16:57:04.687Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d77ddf4c-20f1-4154-8b8c-33006a1ae79b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:57:04.687Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d77ddf4c-20f1-4154-8b8c-33006a1ae79b PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:57:07.599Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=862181c8-fa73-4f22-965d-5321f1ec9095 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:57:07.599Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=862181c8-fa73-4f22-965d-5321f1ec9095 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@33766a5e 2021-08-11T16:57:07.599Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=862181c8-fa73-4f22-965d-5321f1ec9095 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:57:07.599Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=862181c8-fa73-4f22-965d-5321f1ec9095 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:57:10.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=696bbdca-7a0e-480f-ac62-f49caa865548 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:57:10.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=696bbdca-7a0e-480f-ac62-f49caa865548 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@741ecb30 2021-08-11T16:57:10.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=696bbdca-7a0e-480f-ac62-f49caa865548 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:57:10.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=696bbdca-7a0e-480f-ac62-f49caa865548 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:57:13.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72ec454c-eb20-4dbd-808a-b144802bab12 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:57:13.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72ec454c-eb20-4dbd-808a-b144802bab12 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7f1c69dd 2021-08-11T16:57:13.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72ec454c-eb20-4dbd-808a-b144802bab12 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:57:13.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72ec454c-eb20-4dbd-808a-b144802bab12 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:57:16.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce43a6be-6f2e-4135-87fc-ad5fc12ae2bf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:57:16.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce43a6be-6f2e-4135-87fc-ad5fc12ae2bf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@8cc7b44 2021-08-11T16:57:16.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce43a6be-6f2e-4135-87fc-ad5fc12ae2bf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:57:16.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce43a6be-6f2e-4135-87fc-ad5fc12ae2bf PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:57:18.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:57:18.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:57:19.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd1e8fb4-f3a2-4317-86c8-cf9b03bcad45 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:57:19.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd1e8fb4-f3a2-4317-86c8-cf9b03bcad45 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@868c499 2021-08-11T16:57:19.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd1e8fb4-f3a2-4317-86c8-cf9b03bcad45 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:57:19.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd1e8fb4-f3a2-4317-86c8-cf9b03bcad45 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:57:22.654Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=92da072e-613f-4997-bd2c-51294f3c05f4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:57:22.654Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=92da072e-613f-4997-bd2c-51294f3c05f4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@aadd9ef 2021-08-11T16:57:22.654Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=92da072e-613f-4997-bd2c-51294f3c05f4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:57:22.654Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=92da072e-613f-4997-bd2c-51294f3c05f4 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:57:25.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8f1c22dd-a5de-40c3-ac66-412dce8136d3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:57:25.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8f1c22dd-a5de-40c3-ac66-412dce8136d3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@16e30be6 2021-08-11T16:57:25.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8f1c22dd-a5de-40c3-ac66-412dce8136d3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:57:25.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8f1c22dd-a5de-40c3-ac66-412dce8136d3 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:57:28.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fca20eed-7b03-48fd-ba86-37706b9ae931 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:57:28.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fca20eed-7b03-48fd-ba86-37706b9ae931 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@571ade2e 2021-08-11T16:57:28.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fca20eed-7b03-48fd-ba86-37706b9ae931 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:57:28.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fca20eed-7b03-48fd-ba86-37706b9ae931 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:57:29.333Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01nejrrgetn4c31ht5iqze36ej9183 2021-08-11T16:57:29.333Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01ov6hjis22o0astvg490xc4gu194 2021-08-11T16:57:29.333Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node098cs1o5mzn0vrrjxnc40wx5g171 2021-08-11T16:57:29.334Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0qrgues3xgoo4156cbplerpb7p182 2021-08-11T16:57:29.334Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0avfthvvpsqq0l2crscmj9zns186 2021-08-11T16:57:29.334Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0z88x6topzlnjtdb2zy9sm55u191 2021-08-11T16:57:29.334Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0mkegr4dugfni1gcl0jwlqhviq204 2021-08-11T16:57:29.334Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node05nxmzzihplpk8xirkkw8rx8p211 2021-08-11T16:57:29.334Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0xj6z7vizoxqf1ru0z5hhim7bv175 2021-08-11T16:57:29.334Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0j61pccq4ocpdb66p17albv3u198 2021-08-11T16:57:29.334Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0zpyzwkjivnh45goze6jze92k223 2021-08-11T16:57:29.335Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0z5ilrlcvyse91gic9dulr2w3r181 2021-08-11T16:57:29.335Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0yehtcy50x30u1qksm9510eubi173 2021-08-11T16:57:29.335Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0vs7ryvvlqthrvf7mia1lhbf188 2021-08-11T16:57:29.335Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0qamalr4x3ptk1350ngcejw7iq187 2021-08-11T16:57:29.335Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01nqdxcfq7gatx1mbymrtoi4cq8192 2021-08-11T16:57:29.335Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01u4qnvjd5wdrx1mdzz3420e6vw215 2021-08-11T16:57:29.335Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01dg0cbmxmrctvynf83gk624rk189 2021-08-11T16:57:29.335Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node018ckved1weyjmf0cnmwznmv0g213 2021-08-11T16:57:29.335Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01it9u84wxdsx9h25kk25xq0oa203 2021-08-11T16:57:29.335Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01ix8h7ie6mi991gwp4154cy812205 2021-08-11T16:57:29.335Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node07ilbcgo3g3a1rqbrb28yb25h185 2021-08-11T16:57:29.335Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0rf38fdr6hx7l1641pfervqxj2206 2021-08-11T16:57:29.336Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node010rib1bbh73am1nfy26k61mthx217 2021-08-11T16:57:29.336Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0bmcmi9ne7hb413wfy35q8s5ox218 2021-08-11T16:57:29.336Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01ft9mg9sdyphf1kb6rt4h4ys2l174 2021-08-11T16:57:29.336Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node010vcljumlmhpsf8ycv0jln3l1200 2021-08-11T16:57:29.336Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0ex9x7arf4z4qicbj6rqbr270197 2021-08-11T16:57:29.336Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node012jlxd6c98nsc1ntu3cy4tkesx216 2021-08-11T16:57:29.336Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0d7pldmr684dr1t1w7tkro5s7222 2021-08-11T16:57:29.336Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01fsyv61obj5kux0u6rdve7110201 2021-08-11T16:57:29.336Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01a00gcojs7h8h1svpbmcce06we179 2021-08-11T16:57:29.336Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0wmzyc533pary1gom1isch5wwt196 2021-08-11T16:57:29.336Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0xisp05geck3n1676gk7uvqdxh177 2021-08-11T16:57:29.336Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01ozl85s2wfqyf1xaysvif8kjjq190 2021-08-11T16:57:29.336Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0izvd7xc8ibc4vo4fh26da5mv170 2021-08-11T16:57:29.336Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01h5vnjw0tlcc744wad8h9mvtz195 2021-08-11T16:57:29.336Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01digbtmyf08g5qs7d9wy63ros202 2021-08-11T16:57:29.336Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01wyezwmlo4avq1bka42wczsuvz208 2021-08-11T16:57:29.337Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0l0xeey15hdbntvjcn2y0g822176 2021-08-11T16:57:29.337Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0tnlvzqmql8twto20jscmsq0d172 2021-08-11T16:57:29.337Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01y0lyfavr2aju16fugwaa3a7c7180 2021-08-11T16:57:29.337Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01xvxk88728vht4wlzc9240qam184 2021-08-11T16:57:29.337Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01g0n3bcwags8gpa7tl39ibgft214 2021-08-11T16:57:29.337Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01lg5k54348wb61a06wff1qrfdk219 2021-08-11T16:57:29.337Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01qn508s1eca2k4hou30lbjjyr210 2021-08-11T16:57:29.337Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01kfm9u3v70nvj2hok0l8ntmpt193 2021-08-11T16:57:29.337Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0uk881c6getkf11kxqlfiw934d169 2021-08-11T16:57:29.337Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node09zpn9xsw2cnlhty8pt07f3aq207 2021-08-11T16:57:29.337Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0xju6kq925u691hylbnbc60nab212 2021-08-11T16:57:29.337Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01uxv3cbhelhi01h2m3zge0lyh6221 2021-08-11T16:57:29.337Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node04c3wa0x9uuqt1aejmj404v7rx220 2021-08-11T16:57:29.337Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01i7oln0bku10y1cjnvd987ubqj178 2021-08-11T16:57:29.338Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node011dft5o172cbk3yjr559i55rk199 2021-08-11T16:57:31.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5392b3a9-84e1-4f74-8e50-047653c97c71 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:57:31.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5392b3a9-84e1-4f74-8e50-047653c97c71 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@59a84310 2021-08-11T16:57:31.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5392b3a9-84e1-4f74-8e50-047653c97c71 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:57:31.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5392b3a9-84e1-4f74-8e50-047653c97c71 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:57:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:57:33.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:57:34.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4cecadbf-b44a-45df-a9a4-cecf64007f37 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:57:34.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4cecadbf-b44a-45df-a9a4-cecf64007f37 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@56988b6c 2021-08-11T16:57:34.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4cecadbf-b44a-45df-a9a4-cecf64007f37 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:57:34.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4cecadbf-b44a-45df-a9a4-cecf64007f37 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:57:37.751Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4353c19a-45f2-4ec4-a83d-8fe576d31085 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:57:37.751Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4353c19a-45f2-4ec4-a83d-8fe576d31085 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@58fd6af 2021-08-11T16:57:37.751Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4353c19a-45f2-4ec4-a83d-8fe576d31085 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:57:37.751Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4353c19a-45f2-4ec4-a83d-8fe576d31085 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:57:40.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70ac03b5-8a0d-42c4-b69c-68fca0311a4b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:57:40.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70ac03b5-8a0d-42c4-b69c-68fca0311a4b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@465939e 2021-08-11T16:57:40.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70ac03b5-8a0d-42c4-b69c-68fca0311a4b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:57:40.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70ac03b5-8a0d-42c4-b69c-68fca0311a4b PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:57:43.602Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9b2c2ba-b378-4a6d-8b47-5647dd98f75f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:57:43.603Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9b2c2ba-b378-4a6d-8b47-5647dd98f75f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1f4ee0c1 2021-08-11T16:57:43.603Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9b2c2ba-b378-4a6d-8b47-5647dd98f75f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:57:43.603Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9b2c2ba-b378-4a6d-8b47-5647dd98f75f PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:57:46.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8462f5e5-1070-4158-a33c-093c415f403d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:57:46.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8462f5e5-1070-4158-a33c-093c415f403d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@d1fdbaf 2021-08-11T16:57:46.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8462f5e5-1070-4158-a33c-093c415f403d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:57:46.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8462f5e5-1070-4158-a33c-093c415f403d PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:57:48.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:57:48.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:57:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:57:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:57:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:57:50.151Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=118d6b24-baed-48b0-8a40-e608bb3b9385 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:57:50.151Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=118d6b24-baed-48b0-8a40-e608bb3b9385 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@765e4e0d 2021-08-11T16:57:50.151Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=118d6b24-baed-48b0-8a40-e608bb3b9385 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:57:50.152Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=118d6b24-baed-48b0-8a40-e608bb3b9385 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:57:53.151Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64b2a571-6611-4536-a131-f64261d9970f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:57:53.151Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64b2a571-6611-4536-a131-f64261d9970f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5e8fda3b 2021-08-11T16:57:53.151Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64b2a571-6611-4536-a131-f64261d9970f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:57:53.151Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64b2a571-6611-4536-a131-f64261d9970f PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:57:55.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ebefa7d6-c592-4d9a-af70-9a9132f49b94 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:57:55.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ebefa7d6-c592-4d9a-af70-9a9132f49b94 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@362f71c3 2021-08-11T16:57:55.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ebefa7d6-c592-4d9a-af70-9a9132f49b94 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:57:55.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ebefa7d6-c592-4d9a-af70-9a9132f49b94 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:57:58.690Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9559cb85-9a60-407f-be0d-426056efad98 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:57:58.690Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9559cb85-9a60-407f-be0d-426056efad98 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@451e060a 2021-08-11T16:57:58.690Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9559cb85-9a60-407f-be0d-426056efad98 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:57:58.690Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9559cb85-9a60-407f-be0d-426056efad98 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:58:01.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7ee4972d-62f2-4fae-bf2d-d913be42e572 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:58:01.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7ee4972d-62f2-4fae-bf2d-d913be42e572 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@41b40682 2021-08-11T16:58:01.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7ee4972d-62f2-4fae-bf2d-d913be42e572 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:58:01.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7ee4972d-62f2-4fae-bf2d-d913be42e572 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:58:03.305Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:58:03.310Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:58:04.263Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:58:04.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7914fb21-a68d-4a85-9391-39e66ee87855 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:58:04.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7914fb21-a68d-4a85-9391-39e66ee87855 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@61ca428d 2021-08-11T16:58:04.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7914fb21-a68d-4a85-9391-39e66ee87855 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:58:04.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7914fb21-a68d-4a85-9391-39e66ee87855 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:58:07.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1715d02b-345b-4e02-9df3-a39cd61f0827 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:58:07.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1715d02b-345b-4e02-9df3-a39cd61f0827 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2ca6961e 2021-08-11T16:58:07.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1715d02b-345b-4e02-9df3-a39cd61f0827 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:58:07.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1715d02b-345b-4e02-9df3-a39cd61f0827 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:58:10.617Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d69695a2-a154-47fb-b018-84737d95eaba PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:58:10.617Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d69695a2-a154-47fb-b018-84737d95eaba PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@193debd6 2021-08-11T16:58:10.617Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d69695a2-a154-47fb-b018-84737d95eaba PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:58:10.617Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d69695a2-a154-47fb-b018-84737d95eaba PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:58:13.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e570bad3-9523-49b1-a977-8e91ad601227 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:58:13.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e570bad3-9523-49b1-a977-8e91ad601227 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@78dc6e65 2021-08-11T16:58:13.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e570bad3-9523-49b1-a977-8e91ad601227 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:58:13.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e570bad3-9523-49b1-a977-8e91ad601227 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:58:16.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0566204a-74f7-4e1b-bc24-8da7ebcbbc47 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:58:16.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0566204a-74f7-4e1b-bc24-8da7ebcbbc47 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4bb185d9 2021-08-11T16:58:16.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0566204a-74f7-4e1b-bc24-8da7ebcbbc47 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:58:16.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0566204a-74f7-4e1b-bc24-8da7ebcbbc47 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:58:18.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:58:18.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:58:19.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=336d47eb-a9f7-4e7f-9830-878ac0b664f9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:58:19.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=336d47eb-a9f7-4e7f-9830-878ac0b664f9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@53bffcd7 2021-08-11T16:58:19.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=336d47eb-a9f7-4e7f-9830-878ac0b664f9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:58:19.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=336d47eb-a9f7-4e7f-9830-878ac0b664f9 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:58:22.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73c95b6b-5cd3-4247-9baa-df1c5715ef13 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:58:22.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73c95b6b-5cd3-4247-9baa-df1c5715ef13 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@25aace2d 2021-08-11T16:58:22.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73c95b6b-5cd3-4247-9baa-df1c5715ef13 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:58:22.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73c95b6b-5cd3-4247-9baa-df1c5715ef13 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:58:25.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8dcb384a-f26c-4dc6-a3e8-955aeb2475bd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:58:25.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8dcb384a-f26c-4dc6-a3e8-955aeb2475bd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3e1e23bb 2021-08-11T16:58:25.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8dcb384a-f26c-4dc6-a3e8-955aeb2475bd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:58:25.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8dcb384a-f26c-4dc6-a3e8-955aeb2475bd PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:58:28.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79c910e6-b9ec-4453-9efd-95d8ab63a8a8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:58:28.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79c910e6-b9ec-4453-9efd-95d8ab63a8a8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7946f6ec 2021-08-11T16:58:28.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79c910e6-b9ec-4453-9efd-95d8ab63a8a8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:58:28.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79c910e6-b9ec-4453-9efd-95d8ab63a8a8 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:58:31.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c179403-a0a7-4276-82aa-b585c516867d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:58:31.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c179403-a0a7-4276-82aa-b585c516867d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@55b3f457 2021-08-11T16:58:31.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c179403-a0a7-4276-82aa-b585c516867d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:58:31.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c179403-a0a7-4276-82aa-b585c516867d PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:58:33.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:58:33.316Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:58:34.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=962b12a1-bfeb-4d50-a2a3-ef3a628542fd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:58:34.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=962b12a1-bfeb-4d50-a2a3-ef3a628542fd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@261b836a 2021-08-11T16:58:34.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=962b12a1-bfeb-4d50-a2a3-ef3a628542fd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:58:34.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=962b12a1-bfeb-4d50-a2a3-ef3a628542fd PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:58:37.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d1dcb721-08eb-4333-90b6-ca55654fa9d1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:58:37.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d1dcb721-08eb-4333-90b6-ca55654fa9d1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@69d77fe9 2021-08-11T16:58:37.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d1dcb721-08eb-4333-90b6-ca55654fa9d1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:58:37.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d1dcb721-08eb-4333-90b6-ca55654fa9d1 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:58:40.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cdc877cd-c9d2-4c21-aa83-a1c346978ef4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:58:40.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cdc877cd-c9d2-4c21-aa83-a1c346978ef4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@27e805a9 2021-08-11T16:58:40.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cdc877cd-c9d2-4c21-aa83-a1c346978ef4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:58:40.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cdc877cd-c9d2-4c21-aa83-a1c346978ef4 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:58:43.586Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=126340f9-035d-4a92-aa58-b16fbf820582 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:58:43.586Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=126340f9-035d-4a92-aa58-b16fbf820582 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6d335c4f 2021-08-11T16:58:43.586Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=126340f9-035d-4a92-aa58-b16fbf820582 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:58:43.586Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=126340f9-035d-4a92-aa58-b16fbf820582 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:58:46.606Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc96ade7-4643-4323-aefc-3ce940276e47 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:58:46.606Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc96ade7-4643-4323-aefc-3ce940276e47 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@17c99250 2021-08-11T16:58:46.606Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc96ade7-4643-4323-aefc-3ce940276e47 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:58:46.606Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc96ade7-4643-4323-aefc-3ce940276e47 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:58:48.309Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:58:48.316Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:58:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:58:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:58:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:58:50.349Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c5f7c056-5870-4e2e-847b-6866ba0d57bc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:58:50.349Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c5f7c056-5870-4e2e-847b-6866ba0d57bc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@45b38f77 2021-08-11T16:58:50.349Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c5f7c056-5870-4e2e-847b-6866ba0d57bc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:58:50.349Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c5f7c056-5870-4e2e-847b-6866ba0d57bc PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:58:53.249Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d57d2944-5076-4fb2-92be-318cedec243f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:58:53.249Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d57d2944-5076-4fb2-92be-318cedec243f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@22110f25 2021-08-11T16:58:53.249Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d57d2944-5076-4fb2-92be-318cedec243f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:58:53.249Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d57d2944-5076-4fb2-92be-318cedec243f PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:58:56.555Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=46da4559-8e0a-48d1-9255-45c396818589 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:58:56.555Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=46da4559-8e0a-48d1-9255-45c396818589 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@55b7fe0c 2021-08-11T16:58:56.556Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=46da4559-8e0a-48d1-9255-45c396818589 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:58:56.556Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=46da4559-8e0a-48d1-9255-45c396818589 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:58:59.552Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=581c2ec1-59a5-4e63-b981-93832055c78e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:58:59.552Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=581c2ec1-59a5-4e63-b981-93832055c78e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@eb7bdf8 2021-08-11T16:58:59.552Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=581c2ec1-59a5-4e63-b981-93832055c78e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:58:59.552Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=581c2ec1-59a5-4e63-b981-93832055c78e PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:59:01.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa5102d9-9846-4c3e-9c59-faaf4993aedf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:59:01.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa5102d9-9846-4c3e-9c59-faaf4993aedf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@57a6436e 2021-08-11T16:59:01.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa5102d9-9846-4c3e-9c59-faaf4993aedf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:59:01.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa5102d9-9846-4c3e-9c59-faaf4993aedf PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:59:03.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:59:03.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:59:04.244Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:59:04.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b42fc842-6cbd-494b-8317-6c6c80dac9ad PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:59:04.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b42fc842-6cbd-494b-8317-6c6c80dac9ad PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7d641698 2021-08-11T16:59:04.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b42fc842-6cbd-494b-8317-6c6c80dac9ad PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:59:04.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b42fc842-6cbd-494b-8317-6c6c80dac9ad PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:59:07.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=120dece9-2b70-4ee8-b54f-46ae147ccfdd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:59:07.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=120dece9-2b70-4ee8-b54f-46ae147ccfdd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@17c9c0a6 2021-08-11T16:59:07.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=120dece9-2b70-4ee8-b54f-46ae147ccfdd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:59:07.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=120dece9-2b70-4ee8-b54f-46ae147ccfdd PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:59:10.603Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18e2a61b-8958-4305-b856-84d67eb76533 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:59:10.603Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18e2a61b-8958-4305-b856-84d67eb76533 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2422f7da 2021-08-11T16:59:10.603Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18e2a61b-8958-4305-b856-84d67eb76533 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:59:10.603Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18e2a61b-8958-4305-b856-84d67eb76533 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:59:14.062Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0d8a1874-b7e7-49b0-8c94-f013ede3df0d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:59:14.062Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0d8a1874-b7e7-49b0-8c94-f013ede3df0d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@417a7d03 2021-08-11T16:59:14.063Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0d8a1874-b7e7-49b0-8c94-f013ede3df0d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:59:14.063Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0d8a1874-b7e7-49b0-8c94-f013ede3df0d PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:59:16.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=67255264-5dc0-4363-bdbd-262385ab444f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:59:16.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=67255264-5dc0-4363-bdbd-262385ab444f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1cf76362 2021-08-11T16:59:16.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=67255264-5dc0-4363-bdbd-262385ab444f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:59:16.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=67255264-5dc0-4363-bdbd-262385ab444f PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:59:18.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:59:18.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:59:19.617Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=309823a3-834c-4f62-964a-11c56c48fcac PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:59:19.617Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=309823a3-834c-4f62-964a-11c56c48fcac PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@45e24caa 2021-08-11T16:59:19.617Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=309823a3-834c-4f62-964a-11c56c48fcac PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:59:19.617Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=309823a3-834c-4f62-964a-11c56c48fcac PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:59:22.586Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e9420b6-e517-4f17-8017-bb14fd1a94f5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:59:22.586Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e9420b6-e517-4f17-8017-bb14fd1a94f5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@764dffeb 2021-08-11T16:59:22.586Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e9420b6-e517-4f17-8017-bb14fd1a94f5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:59:22.586Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e9420b6-e517-4f17-8017-bb14fd1a94f5 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:59:25.748Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3f6ad8b-dd6b-4b63-a61f-d108172fb125 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:59:25.748Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3f6ad8b-dd6b-4b63-a61f-d108172fb125 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@64cd0644 2021-08-11T16:59:25.748Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3f6ad8b-dd6b-4b63-a61f-d108172fb125 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:59:25.748Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3f6ad8b-dd6b-4b63-a61f-d108172fb125 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:59:28.953Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4790c555-7e96-418c-8086-3a36d4dabab5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:59:28.953Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4790c555-7e96-418c-8086-3a36d4dabab5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@430695cc 2021-08-11T16:59:28.953Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4790c555-7e96-418c-8086-3a36d4dabab5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:59:28.953Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4790c555-7e96-418c-8086-3a36d4dabab5 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:59:31.695Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a98d3310-3e66-4511-b50c-515c016733ff PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:59:31.695Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a98d3310-3e66-4511-b50c-515c016733ff PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@14d6061a 2021-08-11T16:59:31.695Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a98d3310-3e66-4511-b50c-515c016733ff PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:59:31.695Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a98d3310-3e66-4511-b50c-515c016733ff PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:59:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:59:33.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:59:34.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1005a72-deb8-419b-b650-49829da0b56d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:59:34.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1005a72-deb8-419b-b650-49829da0b56d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@d7d11cc 2021-08-11T16:59:34.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1005a72-deb8-419b-b650-49829da0b56d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:59:34.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1005a72-deb8-419b-b650-49829da0b56d PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:59:37.749Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=be2a198a-24bc-4dc3-be2b-f6c5a31c6a4f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:59:37.749Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=be2a198a-24bc-4dc3-be2b-f6c5a31c6a4f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@18ec901 2021-08-11T16:59:37.749Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=be2a198a-24bc-4dc3-be2b-f6c5a31c6a4f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:59:37.750Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=be2a198a-24bc-4dc3-be2b-f6c5a31c6a4f PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:59:40.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c75fe89a-e767-4548-aa67-4431b314b889 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:59:40.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c75fe89a-e767-4548-aa67-4431b314b889 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@76519409 2021-08-11T16:59:40.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c75fe89a-e767-4548-aa67-4431b314b889 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:59:40.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c75fe89a-e767-4548-aa67-4431b314b889 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:59:43.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71a02550-3c5f-4931-aa88-fc51a50575b2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:59:43.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71a02550-3c5f-4931-aa88-fc51a50575b2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5213aa0f 2021-08-11T16:59:43.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71a02550-3c5f-4931-aa88-fc51a50575b2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:59:43.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71a02550-3c5f-4931-aa88-fc51a50575b2 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:59:46.595Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc6299f1-0965-445a-9cd7-9b506dec6922 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:59:46.595Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc6299f1-0965-445a-9cd7-9b506dec6922 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@370532f0 2021-08-11T16:59:46.595Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc6299f1-0965-445a-9cd7-9b506dec6922 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:59:46.595Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc6299f1-0965-445a-9cd7-9b506dec6922 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:59:48.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T16:59:48.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T16:59:49.290Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T16:59:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T16:59:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T16:59:50.055Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed5bca71-c7fc-4863-95c5-2a93ea4070d1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:59:50.055Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed5bca71-c7fc-4863-95c5-2a93ea4070d1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@40b433d0 2021-08-11T16:59:50.055Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed5bca71-c7fc-4863-95c5-2a93ea4070d1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:59:50.055Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ed5bca71-c7fc-4863-95c5-2a93ea4070d1 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:59:53.045Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94a3a421-990e-4a99-804b-36df31f47087 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:59:53.045Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94a3a421-990e-4a99-804b-36df31f47087 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3abf1613 2021-08-11T16:59:53.045Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94a3a421-990e-4a99-804b-36df31f47087 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:59:53.046Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=94a3a421-990e-4a99-804b-36df31f47087 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:59:55.883Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ceec458-6f0c-47e3-b766-0fb061405407 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:59:55.883Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ceec458-6f0c-47e3-b766-0fb061405407 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@75b39099 2021-08-11T16:59:55.883Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ceec458-6f0c-47e3-b766-0fb061405407 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:59:55.883Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ceec458-6f0c-47e3-b766-0fb061405407 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T16:59:58.601Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2dae1a7-a4b9-4673-8c9b-ab24715c2fe3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T16:59:58.602Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2dae1a7-a4b9-4673-8c9b-ab24715c2fe3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6b3fae6d 2021-08-11T16:59:58.602Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2dae1a7-a4b9-4673-8c9b-ab24715c2fe3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T16:59:58.602Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e2dae1a7-a4b9-4673-8c9b-ab24715c2fe3 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:00:01.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37ab5227-26ec-4293-a44a-d6f6fa3ecbbe PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:00:01.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37ab5227-26ec-4293-a44a-d6f6fa3ecbbe PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@af42a69 2021-08-11T17:00:01.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37ab5227-26ec-4293-a44a-d6f6fa3ecbbe PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:00:01.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=37ab5227-26ec-4293-a44a-d6f6fa3ecbbe PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:00:03.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:00:03.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:00:04.257Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:00:04.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eb7a5ff1-e624-4184-980b-21133dd20f75 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:00:04.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eb7a5ff1-e624-4184-980b-21133dd20f75 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@48b9aaaa 2021-08-11T17:00:04.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eb7a5ff1-e624-4184-980b-21133dd20f75 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:00:04.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eb7a5ff1-e624-4184-980b-21133dd20f75 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:00:07.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71623f06-55cc-4faf-8ffa-58a9f0c32196 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:00:07.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71623f06-55cc-4faf-8ffa-58a9f0c32196 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@21e7cc7c 2021-08-11T17:00:07.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71623f06-55cc-4faf-8ffa-58a9f0c32196 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:00:07.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71623f06-55cc-4faf-8ffa-58a9f0c32196 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:00:10.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=182cb686-c89a-4ece-baea-52bf87b81ca3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:00:10.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=182cb686-c89a-4ece-baea-52bf87b81ca3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@37bd42eb 2021-08-11T17:00:10.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=182cb686-c89a-4ece-baea-52bf87b81ca3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:00:10.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=182cb686-c89a-4ece-baea-52bf87b81ca3 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:00:13.602Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55f886ac-6df6-4fe2-902a-b9c6f7cedc76 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:00:13.602Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55f886ac-6df6-4fe2-902a-b9c6f7cedc76 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1cb18117 2021-08-11T17:00:13.602Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55f886ac-6df6-4fe2-902a-b9c6f7cedc76 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:00:13.602Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55f886ac-6df6-4fe2-902a-b9c6f7cedc76 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:00:16.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8b9a4bf-e965-4769-a110-538bd674c241 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:00:16.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8b9a4bf-e965-4769-a110-538bd674c241 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@19d884d1 2021-08-11T17:00:16.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8b9a4bf-e965-4769-a110-538bd674c241 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:00:16.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d8b9a4bf-e965-4769-a110-538bd674c241 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:00:18.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:00:18.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:00:19.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aff7cde0-a23d-4b38-89b8-c5e6be70bc1c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:00:19.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aff7cde0-a23d-4b38-89b8-c5e6be70bc1c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6432fba 2021-08-11T17:00:19.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aff7cde0-a23d-4b38-89b8-c5e6be70bc1c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:00:19.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aff7cde0-a23d-4b38-89b8-c5e6be70bc1c PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:00:22.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=527a2eaf-28d7-4b0d-aa0c-7c8612de46d1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:00:22.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=527a2eaf-28d7-4b0d-aa0c-7c8612de46d1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5b349f83 2021-08-11T17:00:22.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=527a2eaf-28d7-4b0d-aa0c-7c8612de46d1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:00:22.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=527a2eaf-28d7-4b0d-aa0c-7c8612de46d1 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:00:25.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe128bad-9baf-4d23-b6ad-5db18d2594cc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:00:25.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe128bad-9baf-4d23-b6ad-5db18d2594cc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4b9d04cb 2021-08-11T17:00:25.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe128bad-9baf-4d23-b6ad-5db18d2594cc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:00:25.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe128bad-9baf-4d23-b6ad-5db18d2594cc PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:00:28.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd1e76d0-759c-43ae-b438-109bc99fe4e0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:00:28.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd1e76d0-759c-43ae-b438-109bc99fe4e0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@16beef65 2021-08-11T17:00:28.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd1e76d0-759c-43ae-b438-109bc99fe4e0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:00:28.870Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd1e76d0-759c-43ae-b438-109bc99fe4e0 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:00:31.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71b1002f-b782-4621-88b1-7946a56100a7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:00:31.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71b1002f-b782-4621-88b1-7946a56100a7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@99f2cf7 2021-08-11T17:00:31.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71b1002f-b782-4621-88b1-7946a56100a7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:00:31.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71b1002f-b782-4621-88b1-7946a56100a7 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:00:33.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:00:33.321Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:00:34.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=09a8d633-8764-4b8c-963b-29478aef7b52 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:00:34.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=09a8d633-8764-4b8c-963b-29478aef7b52 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@762862c0 2021-08-11T17:00:34.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=09a8d633-8764-4b8c-963b-29478aef7b52 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:00:34.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=09a8d633-8764-4b8c-963b-29478aef7b52 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:00:37.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ede0041-8bba-497b-9dd7-1db061c7ab59 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:00:37.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ede0041-8bba-497b-9dd7-1db061c7ab59 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@55b08b92 2021-08-11T17:00:37.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ede0041-8bba-497b-9dd7-1db061c7ab59 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:00:37.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ede0041-8bba-497b-9dd7-1db061c7ab59 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:00:40.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2298f520-406b-47f3-85ae-7cce57b745f5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:00:40.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2298f520-406b-47f3-85ae-7cce57b745f5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@34948386 2021-08-11T17:00:40.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2298f520-406b-47f3-85ae-7cce57b745f5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:00:40.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2298f520-406b-47f3-85ae-7cce57b745f5 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:00:43.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2736285e-8013-40e2-aa00-5e0c2e04583e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:00:43.633Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2736285e-8013-40e2-aa00-5e0c2e04583e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@449cf977 2021-08-11T17:00:43.633Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2736285e-8013-40e2-aa00-5e0c2e04583e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:00:43.633Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2736285e-8013-40e2-aa00-5e0c2e04583e PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:00:46.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e696f02e-cfa5-4ce5-87f8-a6e13a2b79a3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:00:46.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e696f02e-cfa5-4ce5-87f8-a6e13a2b79a3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7197ad46 2021-08-11T17:00:46.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e696f02e-cfa5-4ce5-87f8-a6e13a2b79a3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:00:46.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e696f02e-cfa5-4ce5-87f8-a6e13a2b79a3 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:00:48.310Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:00:48.318Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:00:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T17:00:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T17:00:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T17:00:50.251Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=498b57b4-0b27-4b14-b47b-3012f5e44cbf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:00:50.251Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=498b57b4-0b27-4b14-b47b-3012f5e44cbf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@20aa54fd 2021-08-11T17:00:50.251Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=498b57b4-0b27-4b14-b47b-3012f5e44cbf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:00:50.251Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=498b57b4-0b27-4b14-b47b-3012f5e44cbf PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:00:53.450Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ade7f35-053b-4e73-8ec6-fb2cd352732a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:00:53.450Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ade7f35-053b-4e73-8ec6-fb2cd352732a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@30327bdd 2021-08-11T17:00:53.450Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ade7f35-053b-4e73-8ec6-fb2cd352732a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:00:53.450Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4ade7f35-053b-4e73-8ec6-fb2cd352732a PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:00:56.749Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1627f64d-b8b2-4721-8938-4b8ca200a614 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:00:56.749Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1627f64d-b8b2-4721-8938-4b8ca200a614 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@13c67bed 2021-08-11T17:00:56.749Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1627f64d-b8b2-4721-8938-4b8ca200a614 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:00:56.749Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1627f64d-b8b2-4721-8938-4b8ca200a614 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:00:59.652Z [cluster1-nio-worker-1] WARN c.d.driver.core.HostConnectionPool Error creating connection to cassandra.onap/10.242.184.131:9042 com.datastax.driver.core.exceptions.TransportException: [cassandra.onap/10.242.184.131:9042] Connection has been closed at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1427) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1408) at com.datastax.driver.core.Connection.defunct(Connection.java:585) at com.datastax.driver.core.Connection$Dispatcher.exceptionCaught(Connection.java:1330) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273) at io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.handleReadException(AbstractEpollStreamChannel.java:728) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:818) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-08-11T17:00:59.653Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.184.131:9042 com.datastax.driver.core.exceptions.ConnectionException: [cassandra.onap/10.242.184.131:9042] Pool was closed during initialization at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:173) at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:158) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1080) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.google.common.util.concurrent.CollectionFuture.handleAllCompleted(CollectionFuture.java:69) at com.google.common.util.concurrent.AggregateFuture.processCompleted(AggregateFuture.java:282) at com.google.common.util.concurrent.AggregateFuture.decrementCountAndMaybeComplete(AggregateFuture.java:264) at com.google.common.util.concurrent.AggregateFuture.access$200(AggregateFuture.java:42) at com.google.common.util.concurrent.AggregateFuture$1.run(AggregateFuture.java:147) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:100) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.datastax.driver.core.Connection$Future.onException(Connection.java:1518) at com.datastax.driver.core.Connection$Dispatcher.errorOutAllHandler(Connection.java:1340) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1427) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1408) at com.datastax.driver.core.Connection.defunct(Connection.java:585) at com.datastax.driver.core.Connection$Dispatcher.exceptionCaught(Connection.java:1330) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273) at io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.handleReadException(AbstractEpollStreamChannel.java:728) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:818) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-08-11T17:00:59.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d80da45e-005a-4f93-92cb-39112f7330fc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:00:59.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d80da45e-005a-4f93-92cb-39112f7330fc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5ff7194e 2021-08-11T17:00:59.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d80da45e-005a-4f93-92cb-39112f7330fc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:00:59.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d80da45e-005a-4f93-92cb-39112f7330fc PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:00:59.870Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=d80da45e-005a-4f93-92cb-39112f7330fc ErrorCategory=INFO ServiceName=BE Component CASSANDRA state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component CASSANDRA state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T17:00:59.870Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=d80da45e-005a-4f93-92cb-39112f7330fc ErrorCategory=ERROR ServiceName=BE Component CASSANDRA state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-08-11T17:01:01.172Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:01.200Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:01.228Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:01.256Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:01.283Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:01.311Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:01.367Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:01.420Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:01.547Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:01.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3a9adae0-2894-403a-9eeb-52b73b87e529 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:01:01.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3a9adae0-2894-403a-9eeb-52b73b87e529 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2d08b3f1 2021-08-11T17:01:01.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3a9adae0-2894-403a-9eeb-52b73b87e529 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:01:01.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3a9adae0-2894-403a-9eeb-52b73b87e529 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:01:01.750Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:01.953Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:02.247Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:02.300Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:02.352Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:02.355Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:02.405Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:02.507Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:02.610Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:02.712Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:02.815Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:03.017Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:03.158Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:03.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:01:03.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-08-11T17:01:03.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:01:03.316Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:01:03.420Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:03.961Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T17:00:55.663211Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:04.227Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:04.258Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:01:04.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1380f096-541a-4f4c-add8-33551202c6d3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:01:04.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1380f096-541a-4f4c-add8-33551202c6d3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@ec7c91e 2021-08-11T17:01:04.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1380f096-541a-4f4c-add8-33551202c6d3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:01:04.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1380f096-541a-4f4c-add8-33551202c6d3 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:01:04.597Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=1380f096-541a-4f4c-add8-33551202c6d3 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T17:01:04.597Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=1380f096-541a-4f4c-add8-33551202c6d3 ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-08-11T17:01:05.030Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:06.245Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:01:06.633Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:07.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8b56f206-2c60-4cd9-b138-b73f2216fbf6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:01:07.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8b56f206-2c60-4cd9-b138-b73f2216fbf6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2efec238 2021-08-11T17:01:07.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8b56f206-2c60-4cd9-b138-b73f2216fbf6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:01:07.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8b56f206-2c60-4cd9-b138-b73f2216fbf6 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:01:08.964Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:08.991Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:09.019Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:09.046Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:09.075Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:09.127Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:09.229Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:09.245Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:01:09.331Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:09.534Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:09.838Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:09.890Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:09.936Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:09.992Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:10.094Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:10.203Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:10.307Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:10.510Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:10.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac0f09da-0e15-40e2-bbd7-4104685f5801 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:01:10.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac0f09da-0e15-40e2-bbd7-4104685f5801 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@49a493c0 2021-08-11T17:01:10.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac0f09da-0e15-40e2-bbd7-4104685f5801 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:01:10.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac0f09da-0e15-40e2-bbd7-4104685f5801 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:01:10.738Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:10.913Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:11.315Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:12.117Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:12.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:01:12.342Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T17:00:55.663211Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:12.920Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:13.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2272b6ee-e276-4e03-afd9-b295061a900d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:01:13.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2272b6ee-e276-4e03-afd9-b295061a900d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@215b363e 2021-08-11T17:01:13.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2272b6ee-e276-4e03-afd9-b295061a900d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:01:13.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2272b6ee-e276-4e03-afd9-b295061a900d PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:01:14.523Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:15.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:01:16.125Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:16.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d9ad4ca8-24af-4513-bb94-5390adeeccce PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:01:16.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d9ad4ca8-24af-4513-bb94-5390adeeccce PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@11e48432 2021-08-11T17:01:16.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d9ad4ca8-24af-4513-bb94-5390adeeccce PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:01:16.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d9ad4ca8-24af-4513-bb94-5390adeeccce PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:01:17.344Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:17.397Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:17.449Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:17.502Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:17.555Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:17.607Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:17.710Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:17.732Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:17.763Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:17.815Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:17.912Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:17.917Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:18.115Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:18.120Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:18.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:01:18.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:01:18.316Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:01:18.318Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:18.523Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:18.722Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:18.926Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:19.124Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:19.328Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:19.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1e2ab8f-9400-4d74-96dd-0af8f3f6dca1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:01:19.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1e2ab8f-9400-4d74-96dd-0af8f3f6dca1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@49d36b34 2021-08-11T17:01:19.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1e2ab8f-9400-4d74-96dd-0af8f3f6dca1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:01:19.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1e2ab8f-9400-4d74-96dd-0af8f3f6dca1 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:01:19.927Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:20.131Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:20.741Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T17:00:55.663211Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:21.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:01:21.735Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:22.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ba922cd-b707-43f8-a8c4-7480534c7acc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:01:22.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ba922cd-b707-43f8-a8c4-7480534c7acc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2484db5d 2021-08-11T17:01:22.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ba922cd-b707-43f8-a8c4-7480534c7acc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:01:22.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1ba922cd-b707-43f8-a8c4-7480534c7acc PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:01:23.338Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:24.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:01:24.941Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:25.748Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:25.756Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c2f46bc3-4bf3-488e-8bf1-65742be2d845 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:01:25.756Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c2f46bc3-4bf3-488e-8bf1-65742be2d845 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5c96e335 2021-08-11T17:01:25.756Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c2f46bc3-4bf3-488e-8bf1-65742be2d845 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:01:25.756Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c2f46bc3-4bf3-488e-8bf1-65742be2d845 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:01:25.800Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:25.903Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:26.048Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:26.150Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:26.353Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:26.550Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:26.648Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:26.748Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:26.755Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:26.851Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:26.952Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:27.155Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:27.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:01:27.357Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:27.557Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:27.559Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:27.761Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:28.164Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:28.948Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f96a3f2-6d36-459b-940e-19d5fa5733e1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:01:28.948Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f96a3f2-6d36-459b-940e-19d5fa5733e1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@69663981 2021-08-11T17:01:28.948Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f96a3f2-6d36-459b-940e-19d5fa5733e1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:01:28.948Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f96a3f2-6d36-459b-940e-19d5fa5733e1 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:01:28.968Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:29.160Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T17:00:55.663211Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:30.250Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:01:30.571Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:31.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0a8887f6-710b-4388-8d13-6fba7725eccd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:01:31.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0a8887f6-710b-4388-8d13-6fba7725eccd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3bbe7552 2021-08-11T17:01:31.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0a8887f6-710b-4388-8d13-6fba7725eccd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:01:31.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0a8887f6-710b-4388-8d13-6fba7725eccd PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:01:32.174Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:33.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:01:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:01:33.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:01:33.777Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:34.163Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:34.216Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:34.268Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:34.320Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:34.373Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:34.425Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:34.477Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:34.579Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:34.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d87d2ac-fca6-412c-8443-fb0831702803 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:01:34.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d87d2ac-fca6-412c-8443-fb0831702803 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@39621f4b 2021-08-11T17:01:34.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d87d2ac-fca6-412c-8443-fb0831702803 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:01:34.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d87d2ac-fca6-412c-8443-fb0831702803 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:01:34.681Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:34.784Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:34.987Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:35.189Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:35.382Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:35.392Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:35.409Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:35.461Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:35.513Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:35.616Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:35.719Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:35.795Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:35.922Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:36.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:01:36.324Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:36.598Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:37.127Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:37.400Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T17:00:55.663211Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:37.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=322c0c2b-71c5-4010-88a8-bf685ca648b1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:01:37.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=322c0c2b-71c5-4010-88a8-bf685ca648b1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5206cc19 2021-08-11T17:01:37.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=322c0c2b-71c5-4010-88a8-bf685ca648b1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:01:37.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=322c0c2b-71c5-4010-88a8-bf685ca648b1 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:01:37.948Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:38.750Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:39.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:01:40.352Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:41.074Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a3ba8ba-3b24-4229-a14c-334d0d1f8cfa PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:01:41.074Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a3ba8ba-3b24-4229-a14c-334d0d1f8cfa PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@af45de9 2021-08-11T17:01:41.074Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a3ba8ba-3b24-4229-a14c-334d0d1f8cfa PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:01:41.074Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a3ba8ba-3b24-4229-a14c-334d0d1f8cfa PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:01:42.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:01:42.403Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:42.455Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:42.508Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:42.560Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:42.662Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:42.864Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:43.067Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:43.269Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:43.472Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:43.556Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:43.584Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:43.601Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e424e0ba-39bf-429d-bcbb-7faff492b4e1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:01:43.601Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e424e0ba-39bf-429d-bcbb-7faff492b4e1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4228ad00 2021-08-11T17:01:43.601Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e424e0ba-39bf-429d-bcbb-7faff492b4e1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:01:43.601Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e424e0ba-39bf-429d-bcbb-7faff492b4e1 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:01:43.610Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:43.647Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:43.699Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:43.808Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:43.875Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:43.911Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:44.014Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:44.117Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:44.278Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:44.319Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:44.522Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:44.681Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:44.724Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:45.127Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:45.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:01:45.483Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T17:00:55.663211Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:45.929Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:01:46.577Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b4c5976-a610-4374-b16a-b48c3707ae4f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:01:46.577Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b4c5976-a610-4374-b16a-b48c3707ae4f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6b196a49 2021-08-11T17:01:46.577Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b4c5976-a610-4374-b16a-b48c3707ae4f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:01:46.577Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b4c5976-a610-4374-b16a-b48c3707ae4f PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:01:47.672Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-08-11T17:01:48.305Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:01:48.318Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:01:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T17:01:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T17:01:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T17:01:49.586Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4cd713ee-f866-4cc4-81f6-a056ba4015bd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:01:49.586Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4cd713ee-f866-4cc4-81f6-a056ba4015bd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@f1b0537 2021-08-11T17:01:49.586Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4cd713ee-f866-4cc4-81f6-a056ba4015bd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:01:49.586Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4cd713ee-f866-4cc4-81f6-a056ba4015bd PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:01:50.351Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=4cd713ee-f866-4cc4-81f6-a056ba4015bd ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T17:01:50.351Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=4cd713ee-f866-4cc4-81f6-a056ba4015bd ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-08-11T17:01:52.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=30761688-f147-44fb-a7b1-e4363eb4755b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:01:52.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=30761688-f147-44fb-a7b1-e4363eb4755b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@27a80a28 2021-08-11T17:01:52.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=30761688-f147-44fb-a7b1-e4363eb4755b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:01:52.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=30761688-f147-44fb-a7b1-e4363eb4755b PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:01:55.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b96118b-6816-445c-9e22-008d95123ab3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:01:55.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b96118b-6816-445c-9e22-008d95123ab3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@206ff370 2021-08-11T17:01:55.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b96118b-6816-445c-9e22-008d95123ab3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:01:55.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3b96118b-6816-445c-9e22-008d95123ab3 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:01:58.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=994156d7-51b0-4627-875f-57a081bd748c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:01:58.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=994156d7-51b0-4627-875f-57a081bd748c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@849e16 2021-08-11T17:01:58.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=994156d7-51b0-4627-875f-57a081bd748c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:01:58.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=994156d7-51b0-4627-875f-57a081bd748c PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:02:03.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:02:03.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:02:04.267Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:02:12.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a28a55f7-ba2e-4ce2-8a08-30aef24703c4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:02:12.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a28a55f7-ba2e-4ce2-8a08-30aef24703c4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5761018c 2021-08-11T17:02:12.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a28a55f7-ba2e-4ce2-8a08-30aef24703c4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:02:12.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a28a55f7-ba2e-4ce2-8a08-30aef24703c4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:02:13.682Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=a28a55f7-ba2e-4ce2-8a08-30aef24703c4 ErrorCategory=INFO ServiceName=BE Component CASSANDRA state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component CASSANDRA state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T17:02:13.683Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=a28a55f7-ba2e-4ce2-8a08-30aef24703c4 ErrorCategory=INFO ServiceName=BE Component CASSANDRA state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T17:02:17.458Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=39a0f9b4-10a9-44b6-8c3c-6c6108860d16 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:02:17.459Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=39a0f9b4-10a9-44b6-8c3c-6c6108860d16 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@307fb802 2021-08-11T17:02:17.459Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=39a0f9b4-10a9-44b6-8c3c-6c6108860d16 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:02:17.459Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=39a0f9b4-10a9-44b6-8c3c-6c6108860d16 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:02:18.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:02:18.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:02:19.065Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5320d78f-9190-41a2-baa5-b3ba5d1c1276 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:02:19.065Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5320d78f-9190-41a2-baa5-b3ba5d1c1276 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6fd4290e 2021-08-11T17:02:19.065Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5320d78f-9190-41a2-baa5-b3ba5d1c1276 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:02:19.065Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5320d78f-9190-41a2-baa5-b3ba5d1c1276 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:02:20.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3ab56513-2396-4aa8-91f5-95fe46ff54a0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:02:20.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3ab56513-2396-4aa8-91f5-95fe46ff54a0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4b046ecc 2021-08-11T17:02:20.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3ab56513-2396-4aa8-91f5-95fe46ff54a0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:02:20.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3ab56513-2396-4aa8-91f5-95fe46ff54a0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:02:22.254Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cc7dffc0-4841-4190-a088-7f19949ab07a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:02:22.255Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cc7dffc0-4841-4190-a088-7f19949ab07a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@14366b41 2021-08-11T17:02:22.255Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cc7dffc0-4841-4190-a088-7f19949ab07a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:02:22.255Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cc7dffc0-4841-4190-a088-7f19949ab07a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:02:23.066Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ecc2c478-59bb-4668-9fea-e27c77985fcb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:02:23.067Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ecc2c478-59bb-4668-9fea-e27c77985fcb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@59c47e6d 2021-08-11T17:02:23.067Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ecc2c478-59bb-4668-9fea-e27c77985fcb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:02:23.067Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ecc2c478-59bb-4668-9fea-e27c77985fcb PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:02:24.079Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=726229f2-3c3b-41e6-ba4e-fa5a0312c3e4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:02:24.079Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=726229f2-3c3b-41e6-ba4e-fa5a0312c3e4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1716b002 2021-08-11T17:02:24.079Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=726229f2-3c3b-41e6-ba4e-fa5a0312c3e4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:02:24.079Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=726229f2-3c3b-41e6-ba4e-fa5a0312c3e4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:02:24.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c2ca430c-fc91-4ff8-8f1f-ae3ce700e01e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:02:24.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c2ca430c-fc91-4ff8-8f1f-ae3ce700e01e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7f0cce22 2021-08-11T17:02:24.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c2ca430c-fc91-4ff8-8f1f-ae3ce700e01e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:02:24.783Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c2ca430c-fc91-4ff8-8f1f-ae3ce700e01e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:02:25.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a70c00aa-e206-463f-9ead-4520612bd1b8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:02:25.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a70c00aa-e206-463f-9ead-4520612bd1b8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@58c7cee1 2021-08-11T17:02:25.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a70c00aa-e206-463f-9ead-4520612bd1b8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:02:25.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a70c00aa-e206-463f-9ead-4520612bd1b8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:02:28.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f99233b5-6e31-4450-8cb7-c107895729d3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:02:28.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f99233b5-6e31-4450-8cb7-c107895729d3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@228af482 2021-08-11T17:02:28.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f99233b5-6e31-4450-8cb7-c107895729d3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:02:28.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f99233b5-6e31-4450-8cb7-c107895729d3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:02:31.689Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7da36f07-fcac-4040-bf22-88661522f395 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:02:31.689Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7da36f07-fcac-4040-bf22-88661522f395 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@bed671 2021-08-11T17:02:31.689Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7da36f07-fcac-4040-bf22-88661522f395 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:02:31.689Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7da36f07-fcac-4040-bf22-88661522f395 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:02:33.309Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:02:33.317Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:02:34.693Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8b4b9498-a601-4728-92c1-36e02dd1e242 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:02:34.693Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8b4b9498-a601-4728-92c1-36e02dd1e242 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@19da759a 2021-08-11T17:02:34.693Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8b4b9498-a601-4728-92c1-36e02dd1e242 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:02:34.693Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8b4b9498-a601-4728-92c1-36e02dd1e242 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:02:37.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=16f6ab7b-a81f-454e-ad00-58ee960ab845 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:02:37.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=16f6ab7b-a81f-454e-ad00-58ee960ab845 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7f8c0086 2021-08-11T17:02:37.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=16f6ab7b-a81f-454e-ad00-58ee960ab845 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:02:37.665Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=16f6ab7b-a81f-454e-ad00-58ee960ab845 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:02:40.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad2e7124-baa8-4516-8347-108ed7228256 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:02:40.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad2e7124-baa8-4516-8347-108ed7228256 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@a57d511 2021-08-11T17:02:40.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad2e7124-baa8-4516-8347-108ed7228256 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:02:40.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad2e7124-baa8-4516-8347-108ed7228256 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:02:43.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b614844f-1ef5-4fc2-afac-256c64b3260a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:02:43.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b614844f-1ef5-4fc2-afac-256c64b3260a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7cac8774 2021-08-11T17:02:43.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b614844f-1ef5-4fc2-afac-256c64b3260a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:02:43.686Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b614844f-1ef5-4fc2-afac-256c64b3260a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:02:46.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1538a67-5148-4570-96fe-541609e47143 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:02:46.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1538a67-5148-4570-96fe-541609e47143 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@36104bde 2021-08-11T17:02:46.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1538a67-5148-4570-96fe-541609e47143 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:02:46.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1538a67-5148-4570-96fe-541609e47143 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:02:48.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:02:48.323Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:02:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T17:02:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T17:02:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T17:02:49.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d45a33d3-c38e-452f-b777-2c0b994d5c40 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:02:49.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d45a33d3-c38e-452f-b777-2c0b994d5c40 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2907f5aa 2021-08-11T17:02:49.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d45a33d3-c38e-452f-b777-2c0b994d5c40 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:02:49.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d45a33d3-c38e-452f-b777-2c0b994d5c40 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:02:52.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b749c97e-73b8-4806-bd77-2809142cee05 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:02:52.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b749c97e-73b8-4806-bd77-2809142cee05 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1d4c42c6 2021-08-11T17:02:52.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b749c97e-73b8-4806-bd77-2809142cee05 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:02:52.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b749c97e-73b8-4806-bd77-2809142cee05 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:02:55.851Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3d748063-e61c-4f6e-a0f3-c0406a7b7ae0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:02:55.851Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3d748063-e61c-4f6e-a0f3-c0406a7b7ae0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@543f13c 2021-08-11T17:02:55.852Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3d748063-e61c-4f6e-a0f3-c0406a7b7ae0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:02:55.852Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3d748063-e61c-4f6e-a0f3-c0406a7b7ae0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:02:58.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ccd362bd-4466-4000-8bcc-f18ca0dc9b05 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:02:58.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ccd362bd-4466-4000-8bcc-f18ca0dc9b05 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@31b914f1 2021-08-11T17:02:58.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ccd362bd-4466-4000-8bcc-f18ca0dc9b05 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:02:58.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ccd362bd-4466-4000-8bcc-f18ca0dc9b05 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:03:01.628Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dacef936-bb39-4f2c-9777-87029b245978 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:03:01.628Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dacef936-bb39-4f2c-9777-87029b245978 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@27851dc9 2021-08-11T17:03:01.628Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dacef936-bb39-4f2c-9777-87029b245978 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:03:01.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dacef936-bb39-4f2c-9777-87029b245978 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:03:03.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:03:03.311Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:03:04.253Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:03:04.730Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd147397-93c9-47c1-8e8e-fcf0e8b987e7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:03:04.731Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd147397-93c9-47c1-8e8e-fcf0e8b987e7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@58a0218d 2021-08-11T17:03:04.731Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd147397-93c9-47c1-8e8e-fcf0e8b987e7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:03:04.731Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd147397-93c9-47c1-8e8e-fcf0e8b987e7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:03:07.687Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8d746e2a-a76d-4063-be12-43bdd45b3e20 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:03:07.687Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8d746e2a-a76d-4063-be12-43bdd45b3e20 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@42f0e1b4 2021-08-11T17:03:07.687Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8d746e2a-a76d-4063-be12-43bdd45b3e20 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:03:07.687Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8d746e2a-a76d-4063-be12-43bdd45b3e20 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:03:10.627Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4e5a7e0-f943-4d90-b944-832adc079a7c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:03:10.627Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4e5a7e0-f943-4d90-b944-832adc079a7c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3b176b87 2021-08-11T17:03:10.627Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4e5a7e0-f943-4d90-b944-832adc079a7c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:03:10.627Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4e5a7e0-f943-4d90-b944-832adc079a7c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:03:13.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=957ff55e-df9e-47d5-8bfd-4431aabf83f5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:03:13.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=957ff55e-df9e-47d5-8bfd-4431aabf83f5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@34f34279 2021-08-11T17:03:13.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=957ff55e-df9e-47d5-8bfd-4431aabf83f5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:03:13.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=957ff55e-df9e-47d5-8bfd-4431aabf83f5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:03:16.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=88bf7dd5-382a-49c9-84b3-edcfd55ce12a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:03:16.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=88bf7dd5-382a-49c9-84b3-edcfd55ce12a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6d86a123 2021-08-11T17:03:16.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=88bf7dd5-382a-49c9-84b3-edcfd55ce12a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:03:16.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=88bf7dd5-382a-49c9-84b3-edcfd55ce12a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:03:18.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:03:18.316Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:03:19.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aca044a1-19c0-41bd-aff5-3dde3924d786 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:03:19.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aca044a1-19c0-41bd-aff5-3dde3924d786 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@28be0066 2021-08-11T17:03:19.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aca044a1-19c0-41bd-aff5-3dde3924d786 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:03:19.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aca044a1-19c0-41bd-aff5-3dde3924d786 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:03:22.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef63800a-554f-4445-9884-e5da753e2582 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:03:22.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef63800a-554f-4445-9884-e5da753e2582 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@28f51da5 2021-08-11T17:03:22.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef63800a-554f-4445-9884-e5da753e2582 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:03:22.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef63800a-554f-4445-9884-e5da753e2582 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:03:25.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=861d7fbf-5c17-43e8-bacd-f88bf293fe48 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:03:25.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=861d7fbf-5c17-43e8-bacd-f88bf293fe48 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6bb7248 2021-08-11T17:03:25.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=861d7fbf-5c17-43e8-bacd-f88bf293fe48 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:03:25.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=861d7fbf-5c17-43e8-bacd-f88bf293fe48 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:03:28.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=48081ed6-6d42-4f77-8a07-47a8e35314e8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:03:28.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=48081ed6-6d42-4f77-8a07-47a8e35314e8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5eec5405 2021-08-11T17:03:28.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=48081ed6-6d42-4f77-8a07-47a8e35314e8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:03:28.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=48081ed6-6d42-4f77-8a07-47a8e35314e8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:03:31.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77a77625-93c3-433d-bc0c-482a131fa705 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:03:31.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77a77625-93c3-433d-bc0c-482a131fa705 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@689d28e 2021-08-11T17:03:31.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77a77625-93c3-433d-bc0c-482a131fa705 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:03:31.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=77a77625-93c3-433d-bc0c-482a131fa705 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:03:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:03:33.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:03:34.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c17f3ece-48d5-4f64-9f27-799ad8a2e9ad PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:03:34.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c17f3ece-48d5-4f64-9f27-799ad8a2e9ad PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@611efcf 2021-08-11T17:03:34.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c17f3ece-48d5-4f64-9f27-799ad8a2e9ad PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:03:34.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c17f3ece-48d5-4f64-9f27-799ad8a2e9ad PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:03:37.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8fea6448-377c-4c8a-9d7d-c06b2f815462 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:03:37.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8fea6448-377c-4c8a-9d7d-c06b2f815462 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5b65f477 2021-08-11T17:03:37.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8fea6448-377c-4c8a-9d7d-c06b2f815462 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:03:37.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8fea6448-377c-4c8a-9d7d-c06b2f815462 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:03:41.047Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b26a53b-9311-4551-9f2c-4d42835bec89 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:03:41.047Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b26a53b-9311-4551-9f2c-4d42835bec89 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6715cf62 2021-08-11T17:03:41.047Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b26a53b-9311-4551-9f2c-4d42835bec89 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:03:41.047Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b26a53b-9311-4551-9f2c-4d42835bec89 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:03:43.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3388d295-ab90-4b44-8854-28a7d09b7571 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:03:43.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3388d295-ab90-4b44-8854-28a7d09b7571 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2fc07cac 2021-08-11T17:03:43.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3388d295-ab90-4b44-8854-28a7d09b7571 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:03:43.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3388d295-ab90-4b44-8854-28a7d09b7571 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:03:46.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=34c0ee86-7d11-4cdb-b7ec-f1e8ac13448d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:03:46.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=34c0ee86-7d11-4cdb-b7ec-f1e8ac13448d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2626910c 2021-08-11T17:03:46.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=34c0ee86-7d11-4cdb-b7ec-f1e8ac13448d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:03:46.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=34c0ee86-7d11-4cdb-b7ec-f1e8ac13448d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:03:48.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:03:48.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:03:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T17:03:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T17:03:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T17:03:50.346Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6fac7855-51a2-4522-91d3-a3ff9ad28ef1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:03:50.346Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6fac7855-51a2-4522-91d3-a3ff9ad28ef1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@52941ff3 2021-08-11T17:03:50.346Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6fac7855-51a2-4522-91d3-a3ff9ad28ef1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:03:50.346Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6fac7855-51a2-4522-91d3-a3ff9ad28ef1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:03:53.174Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0bad9e41-9d3b-452b-aa92-e868403110ba PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:03:53.174Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0bad9e41-9d3b-452b-aa92-e868403110ba PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1b33a41c 2021-08-11T17:03:53.174Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0bad9e41-9d3b-452b-aa92-e868403110ba PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:03:53.174Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0bad9e41-9d3b-452b-aa92-e868403110ba PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:03:56.467Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=536e12e3-3d66-4658-ab21-3a35c3a76d32 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:03:56.467Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=536e12e3-3d66-4658-ab21-3a35c3a76d32 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@37ba8eca 2021-08-11T17:03:56.467Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=536e12e3-3d66-4658-ab21-3a35c3a76d32 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:03:56.467Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=536e12e3-3d66-4658-ab21-3a35c3a76d32 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:03:59.266Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd4e967d-bae6-4d96-a36e-cf173a050d98 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:03:59.266Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd4e967d-bae6-4d96-a36e-cf173a050d98 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2506d7c0 2021-08-11T17:03:59.266Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd4e967d-bae6-4d96-a36e-cf173a050d98 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:03:59.266Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fd4e967d-bae6-4d96-a36e-cf173a050d98 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:04:02.364Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=910e705b-6136-4f5c-8913-fd673fc141b4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:04:02.364Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=910e705b-6136-4f5c-8913-fd673fc141b4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@579f3f14 2021-08-11T17:04:02.364Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=910e705b-6136-4f5c-8913-fd673fc141b4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:04:02.364Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=910e705b-6136-4f5c-8913-fd673fc141b4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:04:03.309Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:04:03.316Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:04:04.255Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:04:04.636Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=336a3528-aec3-4c30-98f5-236a32036755 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:04:04.636Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=336a3528-aec3-4c30-98f5-236a32036755 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@518a72bf 2021-08-11T17:04:04.636Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=336a3528-aec3-4c30-98f5-236a32036755 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:04:04.636Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=336a3528-aec3-4c30-98f5-236a32036755 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:04:07.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73689fd4-32af-4abc-988e-9cdaca2fb60a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:04:07.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73689fd4-32af-4abc-988e-9cdaca2fb60a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@32a57a9a 2021-08-11T17:04:07.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73689fd4-32af-4abc-988e-9cdaca2fb60a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:04:07.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73689fd4-32af-4abc-988e-9cdaca2fb60a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:04:10.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c49919db-ded8-4215-96ce-734703c4efe5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:04:10.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c49919db-ded8-4215-96ce-734703c4efe5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@210e13d9 2021-08-11T17:04:10.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c49919db-ded8-4215-96ce-734703c4efe5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:04:10.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c49919db-ded8-4215-96ce-734703c4efe5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:04:13.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe2a797e-5290-40ca-8cdf-a67bd74a002f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:04:13.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe2a797e-5290-40ca-8cdf-a67bd74a002f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@f47ef73 2021-08-11T17:04:13.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe2a797e-5290-40ca-8cdf-a67bd74a002f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:04:13.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fe2a797e-5290-40ca-8cdf-a67bd74a002f PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:04:16.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=280f5128-50ec-40f0-b004-e9b94c79b860 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:04:16.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=280f5128-50ec-40f0-b004-e9b94c79b860 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@dda7bae 2021-08-11T17:04:16.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=280f5128-50ec-40f0-b004-e9b94c79b860 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:04:16.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=280f5128-50ec-40f0-b004-e9b94c79b860 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:04:18.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:04:18.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:04:19.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f918a369-205f-4740-b336-9e266abea7f2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:04:19.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f918a369-205f-4740-b336-9e266abea7f2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@14b2b125 2021-08-11T17:04:19.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f918a369-205f-4740-b336-9e266abea7f2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:04:19.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f918a369-205f-4740-b336-9e266abea7f2 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:04:22.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8533078-dd26-4b6b-8d80-f954025b520c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:04:22.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8533078-dd26-4b6b-8d80-f954025b520c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4573e8b1 2021-08-11T17:04:22.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8533078-dd26-4b6b-8d80-f954025b520c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:04:22.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c8533078-dd26-4b6b-8d80-f954025b520c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:04:25.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f96431c2-e06f-482b-90bf-bee4f46a2e0b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:04:25.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f96431c2-e06f-482b-90bf-bee4f46a2e0b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@757fbb1b 2021-08-11T17:04:25.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f96431c2-e06f-482b-90bf-bee4f46a2e0b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:04:25.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f96431c2-e06f-482b-90bf-bee4f46a2e0b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:04:28.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6047eb96-8838-485e-bf5a-d73dcbf94184 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:04:28.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6047eb96-8838-485e-bf5a-d73dcbf94184 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2b26a911 2021-08-11T17:04:28.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6047eb96-8838-485e-bf5a-d73dcbf94184 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:04:28.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6047eb96-8838-485e-bf5a-d73dcbf94184 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:04:31.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8e64b6ae-c321-4d23-a1e0-a49312969f61 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:04:31.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8e64b6ae-c321-4d23-a1e0-a49312969f61 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2a1c1ec9 2021-08-11T17:04:31.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8e64b6ae-c321-4d23-a1e0-a49312969f61 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:04:31.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8e64b6ae-c321-4d23-a1e0-a49312969f61 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:04:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:04:33.316Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:04:34.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a856b4f9-fa40-46a5-9c84-73528bee153a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:04:34.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a856b4f9-fa40-46a5-9c84-73528bee153a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@70705d5b 2021-08-11T17:04:34.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a856b4f9-fa40-46a5-9c84-73528bee153a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:04:34.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a856b4f9-fa40-46a5-9c84-73528bee153a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:04:37.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=35a76e45-4912-4a8d-9bf9-0296898b3a79 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:04:37.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=35a76e45-4912-4a8d-9bf9-0296898b3a79 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@78df872d 2021-08-11T17:04:37.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=35a76e45-4912-4a8d-9bf9-0296898b3a79 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:04:37.654Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=35a76e45-4912-4a8d-9bf9-0296898b3a79 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:04:40.719Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f43c07fc-d16c-4fb3-b740-415085142b23 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:04:40.719Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f43c07fc-d16c-4fb3-b740-415085142b23 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@662fd706 2021-08-11T17:04:40.719Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f43c07fc-d16c-4fb3-b740-415085142b23 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:04:40.719Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f43c07fc-d16c-4fb3-b740-415085142b23 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:04:43.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76566768-4027-4b89-98dd-faa8aa5378f5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:04:43.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76566768-4027-4b89-98dd-faa8aa5378f5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2c8f675c 2021-08-11T17:04:43.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76566768-4027-4b89-98dd-faa8aa5378f5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:04:43.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76566768-4027-4b89-98dd-faa8aa5378f5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:04:46.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=829a6b89-004a-416e-b91d-02a2534a4a58 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:04:46.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=829a6b89-004a-416e-b91d-02a2534a4a58 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5e677d13 2021-08-11T17:04:46.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=829a6b89-004a-416e-b91d-02a2534a4a58 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:04:46.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=829a6b89-004a-416e-b91d-02a2534a4a58 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:04:48.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:04:48.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:04:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T17:04:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T17:04:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T17:04:50.451Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3683b473-11b3-4923-9aa4-e76d1d2c44c4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:04:50.451Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3683b473-11b3-4923-9aa4-e76d1d2c44c4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5a857f1d 2021-08-11T17:04:50.451Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3683b473-11b3-4923-9aa4-e76d1d2c44c4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:04:50.451Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3683b473-11b3-4923-9aa4-e76d1d2c44c4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:04:53.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ef86a66-53e2-4f25-8d5c-bb16db3a0abb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:04:53.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ef86a66-53e2-4f25-8d5c-bb16db3a0abb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@420a04ac 2021-08-11T17:04:53.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ef86a66-53e2-4f25-8d5c-bb16db3a0abb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:04:53.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6ef86a66-53e2-4f25-8d5c-bb16db3a0abb PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:04:56.950Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=67aba30e-7f6b-4b8e-8168-0a5a53e2fbb1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:04:56.950Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=67aba30e-7f6b-4b8e-8168-0a5a53e2fbb1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@15e83e36 2021-08-11T17:04:56.950Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=67aba30e-7f6b-4b8e-8168-0a5a53e2fbb1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:04:56.950Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=67aba30e-7f6b-4b8e-8168-0a5a53e2fbb1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:04:59.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cca65c7f-066d-4d2a-8a7f-d6c4b7f0fd0d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:04:59.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cca65c7f-066d-4d2a-8a7f-d6c4b7f0fd0d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7002567d 2021-08-11T17:04:59.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cca65c7f-066d-4d2a-8a7f-d6c4b7f0fd0d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:04:59.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cca65c7f-066d-4d2a-8a7f-d6c4b7f0fd0d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:05:02.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4d565b59-2512-4388-8a75-57044ea1686e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:05:02.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4d565b59-2512-4388-8a75-57044ea1686e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@10eb2f7 2021-08-11T17:05:02.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4d565b59-2512-4388-8a75-57044ea1686e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:05:02.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4d565b59-2512-4388-8a75-57044ea1686e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:05:03.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:05:03.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:05:04.260Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:05:05.477Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=abc015f8-7c09-4af3-90e4-9033458072fb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:05:05.477Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=abc015f8-7c09-4af3-90e4-9033458072fb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1333f91e 2021-08-11T17:05:05.477Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=abc015f8-7c09-4af3-90e4-9033458072fb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:05:05.477Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=abc015f8-7c09-4af3-90e4-9033458072fb PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:05:08.199Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1eb8919a-8eab-41c3-9bd8-155c5c7edf21 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:05:08.199Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1eb8919a-8eab-41c3-9bd8-155c5c7edf21 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@477e6cc0 2021-08-11T17:05:08.199Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1eb8919a-8eab-41c3-9bd8-155c5c7edf21 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:05:08.199Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1eb8919a-8eab-41c3-9bd8-155c5c7edf21 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:05:10.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1f2b5e04-9e0c-466c-85d8-6d50837b242e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:05:10.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1f2b5e04-9e0c-466c-85d8-6d50837b242e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@25ad6bec 2021-08-11T17:05:10.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1f2b5e04-9e0c-466c-85d8-6d50837b242e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:05:10.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1f2b5e04-9e0c-466c-85d8-6d50837b242e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:05:13.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=395cd875-3415-48c4-b3b4-a6573964bce0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:05:13.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=395cd875-3415-48c4-b3b4-a6573964bce0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5140fca4 2021-08-11T17:05:13.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=395cd875-3415-48c4-b3b4-a6573964bce0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:05:13.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=395cd875-3415-48c4-b3b4-a6573964bce0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:05:16.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d450a910-b376-496d-a46f-75cf6805adcb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:05:16.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d450a910-b376-496d-a46f-75cf6805adcb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5d37049b 2021-08-11T17:05:16.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d450a910-b376-496d-a46f-75cf6805adcb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:05:16.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d450a910-b376-496d-a46f-75cf6805adcb PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:05:18.305Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:05:18.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:05:19.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=460fa9ce-9907-40f4-b3b2-bd1be479f1bb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:05:19.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=460fa9ce-9907-40f4-b3b2-bd1be479f1bb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@699b6b91 2021-08-11T17:05:19.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=460fa9ce-9907-40f4-b3b2-bd1be479f1bb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:05:19.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=460fa9ce-9907-40f4-b3b2-bd1be479f1bb PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:05:22.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21b0aa55-0da9-49c6-a569-9054a1c32ac0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:05:22.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21b0aa55-0da9-49c6-a569-9054a1c32ac0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@27279f45 2021-08-11T17:05:22.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21b0aa55-0da9-49c6-a569-9054a1c32ac0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:05:22.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=21b0aa55-0da9-49c6-a569-9054a1c32ac0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:05:25.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7130d955-6a5d-47d3-8555-ed51a67b3236 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:05:25.772Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7130d955-6a5d-47d3-8555-ed51a67b3236 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@71d4e2d9 2021-08-11T17:05:25.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7130d955-6a5d-47d3-8555-ed51a67b3236 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:05:25.773Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7130d955-6a5d-47d3-8555-ed51a67b3236 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:05:28.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba1ea61c-0f9c-4593-ba8c-65b7af13c10c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:05:28.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba1ea61c-0f9c-4593-ba8c-65b7af13c10c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@14a223a4 2021-08-11T17:05:28.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba1ea61c-0f9c-4593-ba8c-65b7af13c10c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:05:28.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ba1ea61c-0f9c-4593-ba8c-65b7af13c10c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:05:31.746Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5dd0ad6b-ace4-4948-9ca0-f2c1e1f708e6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:05:31.746Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5dd0ad6b-ace4-4948-9ca0-f2c1e1f708e6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@26d70542 2021-08-11T17:05:31.746Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5dd0ad6b-ace4-4948-9ca0-f2c1e1f708e6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:05:31.746Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5dd0ad6b-ace4-4948-9ca0-f2c1e1f708e6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:05:33.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:05:33.317Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:05:34.701Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2360429b-1f8d-439d-9b02-377e6e5fdbb5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:05:34.701Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2360429b-1f8d-439d-9b02-377e6e5fdbb5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3d117b6a 2021-08-11T17:05:34.701Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2360429b-1f8d-439d-9b02-377e6e5fdbb5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:05:34.701Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2360429b-1f8d-439d-9b02-377e6e5fdbb5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:05:37.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=15e590b7-679c-4dd5-9aee-5e5c0ab2a371 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:05:37.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=15e590b7-679c-4dd5-9aee-5e5c0ab2a371 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6b05843b 2021-08-11T17:05:37.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=15e590b7-679c-4dd5-9aee-5e5c0ab2a371 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:05:37.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=15e590b7-679c-4dd5-9aee-5e5c0ab2a371 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:05:40.973Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bdb134d8-36f6-4154-8da1-e3f1b4cf1041 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:05:40.973Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bdb134d8-36f6-4154-8da1-e3f1b4cf1041 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6ebf35fc 2021-08-11T17:05:40.973Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bdb134d8-36f6-4154-8da1-e3f1b4cf1041 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:05:40.973Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bdb134d8-36f6-4154-8da1-e3f1b4cf1041 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:05:43.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7f546e51-d3f3-4e57-9e4d-e93f234cd856 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:05:43.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7f546e51-d3f3-4e57-9e4d-e93f234cd856 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@705cd07c 2021-08-11T17:05:43.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7f546e51-d3f3-4e57-9e4d-e93f234cd856 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:05:43.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7f546e51-d3f3-4e57-9e4d-e93f234cd856 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:05:46.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01716202-ead4-4965-a3f8-e9477a72f3b3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:05:46.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01716202-ead4-4965-a3f8-e9477a72f3b3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@46a03f36 2021-08-11T17:05:46.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01716202-ead4-4965-a3f8-e9477a72f3b3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:05:46.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01716202-ead4-4965-a3f8-e9477a72f3b3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:05:48.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:05:48.311Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:05:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T17:05:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T17:05:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T17:05:50.051Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8acdbb08-d050-48a6-9a7a-f1dc1b65de76 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:05:50.051Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8acdbb08-d050-48a6-9a7a-f1dc1b65de76 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1c34f85 2021-08-11T17:05:50.051Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8acdbb08-d050-48a6-9a7a-f1dc1b65de76 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:05:50.051Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8acdbb08-d050-48a6-9a7a-f1dc1b65de76 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:05:53.565Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db629dfc-a423-4a57-a69d-f4a2125e27f1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:05:53.565Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db629dfc-a423-4a57-a69d-f4a2125e27f1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@16448a94 2021-08-11T17:05:53.566Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db629dfc-a423-4a57-a69d-f4a2125e27f1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:05:53.566Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db629dfc-a423-4a57-a69d-f4a2125e27f1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:05:56.371Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d75d2b08-3311-462e-84c8-f6b7c5dab17a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:05:56.371Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d75d2b08-3311-462e-84c8-f6b7c5dab17a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@d7710 2021-08-11T17:05:56.372Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d75d2b08-3311-462e-84c8-f6b7c5dab17a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:05:56.372Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d75d2b08-3311-462e-84c8-f6b7c5dab17a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:05:59.074Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68561af1-f6ce-485e-bd57-3eecc77a0a37 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:05:59.075Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68561af1-f6ce-485e-bd57-3eecc77a0a37 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4713a1a8 2021-08-11T17:05:59.075Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68561af1-f6ce-485e-bd57-3eecc77a0a37 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:05:59.075Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68561af1-f6ce-485e-bd57-3eecc77a0a37 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:06:01.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fcb9037a-67de-4eda-8478-c5c41f06ce07 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:06:01.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fcb9037a-67de-4eda-8478-c5c41f06ce07 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5cb771a8 2021-08-11T17:06:01.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fcb9037a-67de-4eda-8478-c5c41f06ce07 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:06:01.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fcb9037a-67de-4eda-8478-c5c41f06ce07 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:06:03.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:06:03.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:06:04.261Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:06:04.627Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a385f5b9-2961-40f9-97e5-a474b707b759 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:06:04.627Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a385f5b9-2961-40f9-97e5-a474b707b759 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6e154080 2021-08-11T17:06:04.627Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a385f5b9-2961-40f9-97e5-a474b707b759 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:06:04.627Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a385f5b9-2961-40f9-97e5-a474b707b759 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:06:07.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1f8078a3-0a05-44de-9c6b-db4f1931ba51 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:06:07.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1f8078a3-0a05-44de-9c6b-db4f1931ba51 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5cb1fa5 2021-08-11T17:06:07.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1f8078a3-0a05-44de-9c6b-db4f1931ba51 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:06:07.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1f8078a3-0a05-44de-9c6b-db4f1931ba51 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:06:10.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4842a9ad-153c-4f6a-b720-b3c81c5d1186 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:06:10.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4842a9ad-153c-4f6a-b720-b3c81c5d1186 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5f0ff055 2021-08-11T17:06:10.598Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4842a9ad-153c-4f6a-b720-b3c81c5d1186 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:06:10.598Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4842a9ad-153c-4f6a-b720-b3c81c5d1186 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:06:13.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=198a8dba-15a0-49e5-84e5-f3fa9ea14d36 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:06:13.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=198a8dba-15a0-49e5-84e5-f3fa9ea14d36 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@38d2f02d 2021-08-11T17:06:13.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=198a8dba-15a0-49e5-84e5-f3fa9ea14d36 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:06:13.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=198a8dba-15a0-49e5-84e5-f3fa9ea14d36 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:06:16.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=33257c01-f081-4d67-a366-3d89abc57908 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:06:16.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=33257c01-f081-4d67-a366-3d89abc57908 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7c87b6f7 2021-08-11T17:06:16.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=33257c01-f081-4d67-a366-3d89abc57908 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:06:16.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=33257c01-f081-4d67-a366-3d89abc57908 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:06:18.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:06:18.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:06:19.694Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85130ed1-1e56-4663-9f5e-21e3beefd6ee PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:06:19.694Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85130ed1-1e56-4663-9f5e-21e3beefd6ee PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@216ca8f5 2021-08-11T17:06:19.694Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85130ed1-1e56-4663-9f5e-21e3beefd6ee PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:06:19.694Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85130ed1-1e56-4663-9f5e-21e3beefd6ee PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:06:22.687Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e78cd66-d5af-4231-b67d-cd3df5063c41 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:06:22.687Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e78cd66-d5af-4231-b67d-cd3df5063c41 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1c179d5 2021-08-11T17:06:22.687Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e78cd66-d5af-4231-b67d-cd3df5063c41 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:06:22.687Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1e78cd66-d5af-4231-b67d-cd3df5063c41 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:06:25.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=319e56d8-cfed-4002-83ad-2f161b296c91 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:06:25.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=319e56d8-cfed-4002-83ad-2f161b296c91 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2ebdb258 2021-08-11T17:06:25.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=319e56d8-cfed-4002-83ad-2f161b296c91 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:06:25.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=319e56d8-cfed-4002-83ad-2f161b296c91 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:06:28.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0aa8f7d7-5539-4515-9a54-22d9dc963239 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:06:28.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0aa8f7d7-5539-4515-9a54-22d9dc963239 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6f4c8976 2021-08-11T17:06:28.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0aa8f7d7-5539-4515-9a54-22d9dc963239 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:06:28.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0aa8f7d7-5539-4515-9a54-22d9dc963239 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:06:31.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5f5f7db1-8a43-4e3f-b6ac-e645745a04a8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:06:31.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5f5f7db1-8a43-4e3f-b6ac-e645745a04a8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2b23bf6a 2021-08-11T17:06:31.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5f5f7db1-8a43-4e3f-b6ac-e645745a04a8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:06:31.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5f5f7db1-8a43-4e3f-b6ac-e645745a04a8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:06:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:06:33.318Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:06:34.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d9ff4ba2-6c56-45e2-baba-24a93aceb818 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:06:34.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d9ff4ba2-6c56-45e2-baba-24a93aceb818 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2126740a 2021-08-11T17:06:34.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d9ff4ba2-6c56-45e2-baba-24a93aceb818 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:06:34.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d9ff4ba2-6c56-45e2-baba-24a93aceb818 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:06:37.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=07a25e4b-0509-4965-a79b-d7f82871fff6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:06:37.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=07a25e4b-0509-4965-a79b-d7f82871fff6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@62ca3d04 2021-08-11T17:06:37.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=07a25e4b-0509-4965-a79b-d7f82871fff6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:06:37.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=07a25e4b-0509-4965-a79b-d7f82871fff6 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:06:40.633Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=da3f803b-786e-45e9-bb24-f94424eb2c73 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:06:40.633Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=da3f803b-786e-45e9-bb24-f94424eb2c73 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@69460f12 2021-08-11T17:06:40.633Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=da3f803b-786e-45e9-bb24-f94424eb2c73 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:06:40.633Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=da3f803b-786e-45e9-bb24-f94424eb2c73 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:06:43.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c8c98ce-155d-4b17-946d-fe15d676770a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:06:43.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c8c98ce-155d-4b17-946d-fe15d676770a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5f2825bf 2021-08-11T17:06:43.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c8c98ce-155d-4b17-946d-fe15d676770a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:06:43.629Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5c8c98ce-155d-4b17-946d-fe15d676770a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:06:46.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d9535e80-e733-464e-927a-6a2c3ee271fd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:06:46.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d9535e80-e733-464e-927a-6a2c3ee271fd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4d3c8c74 2021-08-11T17:06:46.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d9535e80-e733-464e-927a-6a2c3ee271fd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:06:46.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d9535e80-e733-464e-927a-6a2c3ee271fd PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:06:48.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:06:48.316Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:06:49.290Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T17:06:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T17:06:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T17:06:50.150Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7aa722b-f9fc-4d7a-b76f-fa3a2db3cbd1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:06:50.150Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7aa722b-f9fc-4d7a-b76f-fa3a2db3cbd1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7e2daaca 2021-08-11T17:06:50.150Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7aa722b-f9fc-4d7a-b76f-fa3a2db3cbd1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:06:50.150Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7aa722b-f9fc-4d7a-b76f-fa3a2db3cbd1 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:06:53.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18ce3745-8887-4c5e-94ba-3f449e099eb9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:06:53.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18ce3745-8887-4c5e-94ba-3f449e099eb9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@27ca1482 2021-08-11T17:06:53.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18ce3745-8887-4c5e-94ba-3f449e099eb9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:06:53.761Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18ce3745-8887-4c5e-94ba-3f449e099eb9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:06:56.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=da537edd-2353-4dfe-a135-fe42711d563e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:06:56.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=da537edd-2353-4dfe-a135-fe42711d563e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@39d1c819 2021-08-11T17:06:56.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=da537edd-2353-4dfe-a135-fe42711d563e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:06:56.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=da537edd-2353-4dfe-a135-fe42711d563e PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:07:00.067Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b966d89-0dce-4c46-92b3-54ae2122decc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:07:00.067Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b966d89-0dce-4c46-92b3-54ae2122decc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@10a7bc7b 2021-08-11T17:07:00.067Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b966d89-0dce-4c46-92b3-54ae2122decc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:07:00.067Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b966d89-0dce-4c46-92b3-54ae2122decc PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:07:02.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa10c75d-9940-4afe-a6ed-f28f52d28d19 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:07:02.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa10c75d-9940-4afe-a6ed-f28f52d28d19 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@34f3f5f 2021-08-11T17:07:02.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa10c75d-9940-4afe-a6ed-f28f52d28d19 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:07:02.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aa10c75d-9940-4afe-a6ed-f28f52d28d19 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:07:03.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:07:03.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:07:04.253Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:07:05.280Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=07b754ed-f1ba-45d5-a5c7-ff9f846b4d1b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:07:05.281Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=07b754ed-f1ba-45d5-a5c7-ff9f846b4d1b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@745645b0 2021-08-11T17:07:05.281Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=07b754ed-f1ba-45d5-a5c7-ff9f846b4d1b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:07:05.281Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=07b754ed-f1ba-45d5-a5c7-ff9f846b4d1b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:07:07.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=183ddde5-abbb-4a15-8196-c37611fb18a3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:07:07.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=183ddde5-abbb-4a15-8196-c37611fb18a3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4370ca23 2021-08-11T17:07:07.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=183ddde5-abbb-4a15-8196-c37611fb18a3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:07:07.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=183ddde5-abbb-4a15-8196-c37611fb18a3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:07:10.613Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e7cea125-fb6c-47d5-823c-485bd5485526 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:07:10.613Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e7cea125-fb6c-47d5-823c-485bd5485526 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6e19e252 2021-08-11T17:07:10.613Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e7cea125-fb6c-47d5-823c-485bd5485526 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:07:10.613Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e7cea125-fb6c-47d5-823c-485bd5485526 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:07:13.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa0cfc78-9963-4a25-b4ce-aa6154f3ee1c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:07:13.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa0cfc78-9963-4a25-b4ce-aa6154f3ee1c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@14569d09 2021-08-11T17:07:13.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa0cfc78-9963-4a25-b4ce-aa6154f3ee1c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:07:13.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa0cfc78-9963-4a25-b4ce-aa6154f3ee1c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:07:17.263Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d2b5a853-b1e4-48be-af20-18db0fda1ab0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:07:17.264Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d2b5a853-b1e4-48be-af20-18db0fda1ab0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@526b31f6 2021-08-11T17:07:17.264Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d2b5a853-b1e4-48be-af20-18db0fda1ab0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:07:17.264Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d2b5a853-b1e4-48be-af20-18db0fda1ab0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:07:18.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:07:18.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:07:19.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d9c26fa-e079-4837-9b18-6db90527e84a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:07:19.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d9c26fa-e079-4837-9b18-6db90527e84a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@178fd84a 2021-08-11T17:07:19.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d9c26fa-e079-4837-9b18-6db90527e84a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:07:19.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9d9c26fa-e079-4837-9b18-6db90527e84a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:07:22.694Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d33c7827-bd06-4f4d-8955-182adeeca77b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:07:22.694Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d33c7827-bd06-4f4d-8955-182adeeca77b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3625a39b 2021-08-11T17:07:22.694Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d33c7827-bd06-4f4d-8955-182adeeca77b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:07:22.695Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d33c7827-bd06-4f4d-8955-182adeeca77b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:07:25.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=855689b5-d1a2-42eb-97a7-8d76e9389609 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:07:25.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=855689b5-d1a2-42eb-97a7-8d76e9389609 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@16bbde83 2021-08-11T17:07:25.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=855689b5-d1a2-42eb-97a7-8d76e9389609 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:07:25.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=855689b5-d1a2-42eb-97a7-8d76e9389609 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:07:28.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b827ccc-b00b-485a-8fe2-da302a4c4935 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:07:28.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b827ccc-b00b-485a-8fe2-da302a4c4935 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1dd4a923 2021-08-11T17:07:28.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b827ccc-b00b-485a-8fe2-da302a4c4935 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:07:28.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b827ccc-b00b-485a-8fe2-da302a4c4935 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:07:31.721Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0cc7243-4477-42ba-aff9-5514fe6c699d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:07:31.721Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0cc7243-4477-42ba-aff9-5514fe6c699d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@74f06f5b 2021-08-11T17:07:31.721Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0cc7243-4477-42ba-aff9-5514fe6c699d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:07:31.721Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0cc7243-4477-42ba-aff9-5514fe6c699d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:07:33.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:07:33.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:07:34.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5943193a-d0e0-45f3-8799-9acf6353290c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:07:34.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5943193a-d0e0-45f3-8799-9acf6353290c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6f522325 2021-08-11T17:07:34.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5943193a-d0e0-45f3-8799-9acf6353290c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:07:34.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5943193a-d0e0-45f3-8799-9acf6353290c PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:07:37.690Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b4e132be-08ec-4fd1-bbb8-5b0d70c97a26 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:07:37.690Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b4e132be-08ec-4fd1-bbb8-5b0d70c97a26 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2755b78a 2021-08-11T17:07:37.690Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b4e132be-08ec-4fd1-bbb8-5b0d70c97a26 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:07:37.690Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b4e132be-08ec-4fd1-bbb8-5b0d70c97a26 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:07:40.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4280a3ca-a130-4ae5-8e58-53d724eed374 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:07:40.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4280a3ca-a130-4ae5-8e58-53d724eed374 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1f708ff2 2021-08-11T17:07:40.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4280a3ca-a130-4ae5-8e58-53d724eed374 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:07:40.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4280a3ca-a130-4ae5-8e58-53d724eed374 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:07:43.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b4aa9460-08b9-43e0-a7cb-60308439af24 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:07:43.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b4aa9460-08b9-43e0-a7cb-60308439af24 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@680db1f0 2021-08-11T17:07:43.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b4aa9460-08b9-43e0-a7cb-60308439af24 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:07:43.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b4aa9460-08b9-43e0-a7cb-60308439af24 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:07:46.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f98b8b8-29e0-46c9-84e2-f0879faa1816 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:07:46.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f98b8b8-29e0-46c9-84e2-f0879faa1816 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1638ecdf 2021-08-11T17:07:46.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f98b8b8-29e0-46c9-84e2-f0879faa1816 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:07:46.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4f98b8b8-29e0-46c9-84e2-f0879faa1816 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:07:48.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:07:48.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:07:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T17:07:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T17:07:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T17:07:50.245Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac2f692c-b438-4ea4-acf2-910a3e436f29 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:07:50.245Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac2f692c-b438-4ea4-acf2-910a3e436f29 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1cdbf075 2021-08-11T17:07:50.245Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac2f692c-b438-4ea4-acf2-910a3e436f29 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:07:50.245Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac2f692c-b438-4ea4-acf2-910a3e436f29 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:07:53.161Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e686b086-8b8f-418b-a587-6578be473666 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:07:53.161Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e686b086-8b8f-418b-a587-6578be473666 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@74ff303 2021-08-11T17:07:53.161Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e686b086-8b8f-418b-a587-6578be473666 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:07:53.161Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e686b086-8b8f-418b-a587-6578be473666 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:07:56.364Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=957b39f6-7939-4715-aba2-7de09c5796d0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:07:56.364Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=957b39f6-7939-4715-aba2-7de09c5796d0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@248402b 2021-08-11T17:07:56.364Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=957b39f6-7939-4715-aba2-7de09c5796d0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:07:56.364Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=957b39f6-7939-4715-aba2-7de09c5796d0 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:07:59.129Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e73013cb-379f-4666-b0dd-48af93c3cbc5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:07:59.129Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e73013cb-379f-4666-b0dd-48af93c3cbc5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@53471aa9 2021-08-11T17:07:59.129Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e73013cb-379f-4666-b0dd-48af93c3cbc5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:07:59.129Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e73013cb-379f-4666-b0dd-48af93c3cbc5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:08:01.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66df4ff6-ebf0-44b3-81f4-121169492fd7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:08:01.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66df4ff6-ebf0-44b3-81f4-121169492fd7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@287a6222 2021-08-11T17:08:01.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66df4ff6-ebf0-44b3-81f4-121169492fd7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:08:01.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66df4ff6-ebf0-44b3-81f4-121169492fd7 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:08:03.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:08:03.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:08:04.267Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:08:04.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66373cd8-633d-4970-87e8-3a587a35f948 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:08:04.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66373cd8-633d-4970-87e8-3a587a35f948 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4303a7d6 2021-08-11T17:08:04.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66373cd8-633d-4970-87e8-3a587a35f948 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:08:04.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66373cd8-633d-4970-87e8-3a587a35f948 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:08:07.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8d3d6253-a660-4887-99ed-9630874647c3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:08:07.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8d3d6253-a660-4887-99ed-9630874647c3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3328843a 2021-08-11T17:08:07.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8d3d6253-a660-4887-99ed-9630874647c3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:08:07.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8d3d6253-a660-4887-99ed-9630874647c3 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:08:10.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8afff64d-1a5a-4711-b4ef-42bfd7b7b483 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:08:10.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8afff64d-1a5a-4711-b4ef-42bfd7b7b483 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@42da4706 2021-08-11T17:08:10.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8afff64d-1a5a-4711-b4ef-42bfd7b7b483 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:08:10.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8afff64d-1a5a-4711-b4ef-42bfd7b7b483 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:08:13.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7910f451-8b3e-43fd-8c29-08373967f5eb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:08:13.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7910f451-8b3e-43fd-8c29-08373967f5eb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@394a460d 2021-08-11T17:08:13.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7910f451-8b3e-43fd-8c29-08373967f5eb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:08:13.666Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7910f451-8b3e-43fd-8c29-08373967f5eb PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:08:16.689Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4985ca77-c091-4b67-bdce-5b32bfd84bef PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:08:16.689Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4985ca77-c091-4b67-bdce-5b32bfd84bef PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@67a96be5 2021-08-11T17:08:16.689Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4985ca77-c091-4b67-bdce-5b32bfd84bef PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:08:16.689Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4985ca77-c091-4b67-bdce-5b32bfd84bef PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:08:18.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:08:18.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:08:19.704Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=081a04e4-8a5b-45e7-8132-85cabd4b5c1d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:08:19.704Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=081a04e4-8a5b-45e7-8132-85cabd4b5c1d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@10572a7e 2021-08-11T17:08:19.704Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=081a04e4-8a5b-45e7-8132-85cabd4b5c1d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:08:19.704Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=081a04e4-8a5b-45e7-8132-85cabd4b5c1d PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:08:22.690Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8c969809-3187-46ab-8ec3-1756aa72c9bf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:08:22.690Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8c969809-3187-46ab-8ec3-1756aa72c9bf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@a5bd05 2021-08-11T17:08:22.691Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8c969809-3187-46ab-8ec3-1756aa72c9bf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:08:22.691Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8c969809-3187-46ab-8ec3-1756aa72c9bf PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:08:25.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d192c171-bded-4d3b-9256-57bf22a84c94 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:08:25.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d192c171-bded-4d3b-9256-57bf22a84c94 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6bb4e932 2021-08-11T17:08:25.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d192c171-bded-4d3b-9256-57bf22a84c94 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:08:25.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d192c171-bded-4d3b-9256-57bf22a84c94 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:08:28.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18423d2a-1717-458f-bb2f-5b7c2fa52a05 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:08:28.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18423d2a-1717-458f-bb2f-5b7c2fa52a05 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@47ee51e2 2021-08-11T17:08:28.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18423d2a-1717-458f-bb2f-5b7c2fa52a05 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:08:28.769Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18423d2a-1717-458f-bb2f-5b7c2fa52a05 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:08:29.339Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0gso8645ify1fujohhz4024z5228 2021-08-11T17:08:29.339Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node091iqpdtk2uqo1b3mh6rbv282n235 2021-08-11T17:08:29.340Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01w945o7nw733x1v3ydx36rok7g231 2021-08-11T17:08:29.340Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0wqq66zk42vzg1jb9bscc1dmbg243 2021-08-11T17:08:29.340Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0dxmjl59p73aa1w53z7e9pspkt225 2021-08-11T17:08:29.340Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0wnkfa1xi3725uk4qiccx2qrz242 2021-08-11T17:08:29.340Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0y89qsdeazk4oppbero7y4jp3236 2021-08-11T17:08:29.340Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node076jxg14npgne1ll0ntajfzx0229 2021-08-11T17:08:29.340Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0io8fwga106edojczht1hpdn2232 2021-08-11T17:08:29.340Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0squfx1oik4drbguwjllnnb1v226 2021-08-11T17:08:29.340Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node03pxlybmlvc9c1olq54cx2ie7i245 2021-08-11T17:08:29.341Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01c3me4h5sg8hnhz8qvhj9mgn2230 2021-08-11T17:08:29.341Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01g4idcswbdx881btajc82c8i9f237 2021-08-11T17:08:29.341Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node016l7c4pajnof94m10h58qvu5x240 2021-08-11T17:08:29.341Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01i5cyk5klheuks5tkhapx9p5z224 2021-08-11T17:08:29.341Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0fz3pwv6uqulydpesao4dutgz239 2021-08-11T17:08:29.341Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node08gbu5qqorjck1obuqsz0ehf46233 2021-08-11T17:08:29.341Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node026xhhxyv87951u36htd07a215234 2021-08-11T17:08:29.341Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0mm10tp7zcuux1t463tyya18rp241 2021-08-11T17:08:29.341Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0161u4kzwvrrha1vllqjwskgq78227 2021-08-11T17:08:29.341Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01iqhnkr9m3zujbhjwieu63hyk244 2021-08-11T17:08:29.341Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01hc1rk24dmb1gzgr37o186sgr238 2021-08-11T17:08:31.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b206761b-7f83-4607-8a02-1dc50803b56b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:08:31.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b206761b-7f83-4607-8a02-1dc50803b56b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6e4397bb 2021-08-11T17:08:31.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b206761b-7f83-4607-8a02-1dc50803b56b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:08:31.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b206761b-7f83-4607-8a02-1dc50803b56b PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:08:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:08:33.317Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:08:34.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a1756fb5-d64c-4de9-b1f9-9967153f3096 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:08:34.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a1756fb5-d64c-4de9-b1f9-9967153f3096 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@432342e3 2021-08-11T17:08:34.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a1756fb5-d64c-4de9-b1f9-9967153f3096 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:08:34.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a1756fb5-d64c-4de9-b1f9-9967153f3096 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:08:37.711Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eae08fba-424f-4e08-bce4-f379ccb22ea8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:08:37.712Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eae08fba-424f-4e08-bce4-f379ccb22ea8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@f12cecc 2021-08-11T17:08:37.712Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eae08fba-424f-4e08-bce4-f379ccb22ea8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:08:37.712Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=eae08fba-424f-4e08-bce4-f379ccb22ea8 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:08:40.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3632a689-8f8e-47ab-98b9-2980ef62bdd5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:08:40.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3632a689-8f8e-47ab-98b9-2980ef62bdd5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3ac6ff8d 2021-08-11T17:08:40.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3632a689-8f8e-47ab-98b9-2980ef62bdd5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:08:40.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3632a689-8f8e-47ab-98b9-2980ef62bdd5 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:08:43.641Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=570cc95e-a91d-49be-956c-2a3bf2205246 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:08:43.641Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=570cc95e-a91d-49be-956c-2a3bf2205246 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@484a4921 2021-08-11T17:08:43.641Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=570cc95e-a91d-49be-956c-2a3bf2205246 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:08:43.641Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=570cc95e-a91d-49be-956c-2a3bf2205246 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:08:46.690Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef3bee35-675d-4332-99f3-90e39d329eb4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:08:46.690Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef3bee35-675d-4332-99f3-90e39d329eb4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@11272e91 2021-08-11T17:08:46.690Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef3bee35-675d-4332-99f3-90e39d329eb4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:08:46.690Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef3bee35-675d-4332-99f3-90e39d329eb4 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:08:48.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:08:48.311Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:08:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T17:08:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T17:08:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T17:08:50.255Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3c17731-9693-4474-b050-d473a5804dee PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:08:50.255Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3c17731-9693-4474-b050-d473a5804dee PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@85ae6d7 2021-08-11T17:08:50.255Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3c17731-9693-4474-b050-d473a5804dee PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:08:50.255Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f3c17731-9693-4474-b050-d473a5804dee PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:08:53.873Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=673b7352-28c6-445c-a380-ff07c176a4c9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:08:53.873Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=673b7352-28c6-445c-a380-ff07c176a4c9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@56d4ac6 2021-08-11T17:08:53.873Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=673b7352-28c6-445c-a380-ff07c176a4c9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:08:53.873Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=673b7352-28c6-445c-a380-ff07c176a4c9 PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:08:56.965Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76a18565-2444-4b68-99cd-48c7d53bbd8a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:08:56.965Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76a18565-2444-4b68-99cd-48c7d53bbd8a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1f74080b 2021-08-11T17:08:56.965Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76a18565-2444-4b68-99cd-48c7d53bbd8a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:08:56.965Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=76a18565-2444-4b68-99cd-48c7d53bbd8a PartnerName=SDC.BE The cassandra down nodes number is 0 2021-08-11T17:08:59.257Z [cluster1-nio-worker-1] WARN c.d.driver.core.HostConnectionPool Error creating connection to cassandra.onap/10.242.184.131:9042 com.datastax.driver.core.exceptions.TransportException: [cassandra.onap/10.242.184.131:9042] Connection has been closed at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1427) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1408) at com.datastax.driver.core.Connection.defunct(Connection.java:585) at com.datastax.driver.core.Connection$Dispatcher.exceptionCaught(Connection.java:1330) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273) at io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.handleReadException(AbstractEpollStreamChannel.java:728) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:818) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-08-11T17:08:59.257Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.184.131:9042 com.datastax.driver.core.exceptions.ConnectionException: [cassandra.onap/10.242.184.131:9042] Pool was closed during initialization at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:173) at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:158) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1080) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.google.common.util.concurrent.CollectionFuture.handleAllCompleted(CollectionFuture.java:69) at com.google.common.util.concurrent.AggregateFuture.processCompleted(AggregateFuture.java:282) at com.google.common.util.concurrent.AggregateFuture.decrementCountAndMaybeComplete(AggregateFuture.java:264) at com.google.common.util.concurrent.AggregateFuture.access$200(AggregateFuture.java:42) at com.google.common.util.concurrent.AggregateFuture$1.run(AggregateFuture.java:147) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:100) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.datastax.driver.core.Connection$Future.onException(Connection.java:1518) at com.datastax.driver.core.Connection$Dispatcher.errorOutAllHandler(Connection.java:1340) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1427) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1408) at com.datastax.driver.core.Connection.defunct(Connection.java:585) at com.datastax.driver.core.Connection$Dispatcher.exceptionCaught(Connection.java:1330) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273) at io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.handleReadException(AbstractEpollStreamChannel.java:728) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:818) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-08-11T17:08:59.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce97021b-a23c-4a34-bc7a-b54b7d136d15 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:08:59.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce97021b-a23c-4a34-bc7a-b54b7d136d15 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@370ffed2 2021-08-11T17:08:59.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce97021b-a23c-4a34-bc7a-b54b7d136d15 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:08:59.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce97021b-a23c-4a34-bc7a-b54b7d136d15 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:08:59.976Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=ce97021b-a23c-4a34-bc7a-b54b7d136d15 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T17:08:59.976Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=ce97021b-a23c-4a34-bc7a-b54b7d136d15 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T17:09:01.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5fa47d16-0af3-4f68-b16f-d9234c8f6f79 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:09:01.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5fa47d16-0af3-4f68-b16f-d9234c8f6f79 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@68bccaba 2021-08-11T17:09:01.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5fa47d16-0af3-4f68-b16f-d9234c8f6f79 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:09:01.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5fa47d16-0af3-4f68-b16f-d9234c8f6f79 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:09:01.591Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=5fa47d16-0af3-4f68-b16f-d9234c8f6f79 ErrorCategory=INFO ServiceName=BE Component CASSANDRA state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component CASSANDRA state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T17:09:01.592Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=5fa47d16-0af3-4f68-b16f-d9234c8f6f79 ErrorCategory=ERROR ServiceName=BE Component CASSANDRA state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-08-11T17:09:01.666Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:01.692Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:01.719Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:01.771Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:01.824Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:01.876Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:01.978Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:02.081Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:02.247Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:02.284Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:02.299Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:02.351Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:02.403Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:02.505Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:02.607Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:02.686Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:02.709Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:02.912Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:03.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:09:03.245Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-08-11T17:09:03.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:09:03.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:09:03.314Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:03.489Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:03.719Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:04.244Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:09:04.522Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:04.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=966a7f8b-da33-4778-9d30-b4a9f53e9160 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:09:04.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=966a7f8b-da33-4778-9d30-b4a9f53e9160 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7e89611 2021-08-11T17:09:04.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=966a7f8b-da33-4778-9d30-b4a9f53e9160 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:09:04.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=966a7f8b-da33-4778-9d30-b4a9f53e9160 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:09:04.595Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=966a7f8b-da33-4778-9d30-b4a9f53e9160 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T17:09:04.596Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=966a7f8b-da33-4778-9d30-b4a9f53e9160 ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-08-11T17:09:05.092Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T17:08:56.120689Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:06.124Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:06.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:09:07.594Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc6b823a-c243-4efa-a0df-cb787e879a25 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:09:07.595Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc6b823a-c243-4efa-a0df-cb787e879a25 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3863e2b7 2021-08-11T17:09:07.595Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc6b823a-c243-4efa-a0df-cb787e879a25 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:09:07.595Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc6b823a-c243-4efa-a0df-cb787e879a25 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:09:09.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:09:09.328Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:09.380Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:09.483Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:09.685Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:09.888Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:10.090Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:10.093Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:10.120Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:10.173Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:10.226Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:10.279Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:10.293Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:10.331Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:10.434Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:10.495Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:10.587Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5eda5a88-c0a9-4a1f-ac5f-24cc6bf6fad1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:09:10.587Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5eda5a88-c0a9-4a1f-ac5f-24cc6bf6fad1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@70d27207 2021-08-11T17:09:10.587Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5eda5a88-c0a9-4a1f-ac5f-24cc6bf6fad1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:09:10.587Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5eda5a88-c0a9-4a1f-ac5f-24cc6bf6fad1 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:09:10.636Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:10.902Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:11.039Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:11.704Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:11.841Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:12.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:09:12.644Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:13.307Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:13.447Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T17:08:56.120689Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:13.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=afb4eb82-b4c1-4b5f-a72e-181ea248d0a2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:09:13.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=afb4eb82-b4c1-4b5f-a72e-181ea248d0a2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7e358fbd 2021-08-11T17:09:13.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=afb4eb82-b4c1-4b5f-a72e-181ea248d0a2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:09:13.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=afb4eb82-b4c1-4b5f-a72e-181ea248d0a2 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:09:14.910Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:15.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:09:16.513Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:16.590Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0fd315b-c479-4400-b24f-014c7d224106 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:09:16.590Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0fd315b-c479-4400-b24f-014c7d224106 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3daec674 2021-08-11T17:09:16.590Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0fd315b-c479-4400-b24f-014c7d224106 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:09:16.590Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0fd315b-c479-4400-b24f-014c7d224106 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:09:18.117Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:18.170Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:18.222Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:18.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:09:18.275Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:18.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:09:18.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:09:18.327Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:18.379Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:18.431Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:18.449Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:18.477Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:18.484Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:18.504Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:18.532Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:18.536Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:18.585Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:18.588Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:18.687Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:18.690Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:18.890Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:18.893Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:19.095Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:19.294Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:19.298Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:19.501Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:19.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a65b9cb-0c5e-4373-bf8b-5a7754967530 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:09:19.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a65b9cb-0c5e-4373-bf8b-5a7754967530 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1afffd2c 2021-08-11T17:09:19.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a65b9cb-0c5e-4373-bf8b-5a7754967530 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:09:19.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2a65b9cb-0c5e-4373-bf8b-5a7754967530 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:09:19.704Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:19.907Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:20.096Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:20.310Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:20.713Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:21.116Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:21.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:09:21.519Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:21.699Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T17:08:56.120689Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:21.921Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:22.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4d884984-8ea5-45dd-831c-c9aeb41cbc39 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:09:22.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4d884984-8ea5-45dd-831c-c9aeb41cbc39 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@795ee63e 2021-08-11T17:09:22.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4d884984-8ea5-45dd-831c-c9aeb41cbc39 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:09:22.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4d884984-8ea5-45dd-831c-c9aeb41cbc39 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:09:22.724Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:24.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:09:24.327Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:25.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f248c9de-4d6a-4e17-ae01-088de4d0da86 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:09:25.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f248c9de-4d6a-4e17-ae01-088de4d0da86 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2bc4c6cf 2021-08-11T17:09:25.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f248c9de-4d6a-4e17-ae01-088de4d0da86 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:09:25.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f248c9de-4d6a-4e17-ae01-088de4d0da86 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:09:26.748Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:26.775Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:26.847Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:26.950Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:27.052Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:27.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:09:27.255Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:27.458Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:27.549Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:27.576Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:27.647Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:27.660Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:27.747Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:27.847Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:27.950Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:28.062Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:28.152Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:28.355Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:28.758Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:28.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=363d86e8-27e1-420d-8246-2c56e58ed1c0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:09:28.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=363d86e8-27e1-420d-8246-2c56e58ed1c0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@609228a5 2021-08-11T17:09:28.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=363d86e8-27e1-420d-8246-2c56e58ed1c0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:09:28.862Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=363d86e8-27e1-420d-8246-2c56e58ed1c0 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:09:28.865Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:29.560Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:30.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:09:30.468Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T17:08:56.120689Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:31.167Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:31.587Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b52ba5d9-8942-4b55-8771-be9080133c6b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:09:31.587Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b52ba5d9-8942-4b55-8771-be9080133c6b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5e1af06 2021-08-11T17:09:31.587Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b52ba5d9-8942-4b55-8771-be9080133c6b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:09:31.587Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b52ba5d9-8942-4b55-8771-be9080133c6b PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:09:32.769Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:33.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:09:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:09:33.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:09:34.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c79e937-45e7-41c9-8241-355a68eeda91 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:09:34.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c79e937-45e7-41c9-8241-355a68eeda91 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@b00d692 2021-08-11T17:09:34.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c79e937-45e7-41c9-8241-355a68eeda91 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:09:34.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6c79e937-45e7-41c9-8241-355a68eeda91 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:09:35.471Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:35.523Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:35.575Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:35.628Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:35.731Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:35.933Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:35.973Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:36.025Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:36.127Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:36.135Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:36.248Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:36.252Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:09:36.451Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:36.538Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:36.854Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:36.940Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:37.257Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:37.345Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:37.659Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:37.747Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:37.759Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70533f70-e19b-4768-9017-9029f53a6454 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:09:37.759Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70533f70-e19b-4768-9017-9029f53a6454 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@67f8b83 2021-08-11T17:09:37.759Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70533f70-e19b-4768-9017-9029f53a6454 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:09:37.759Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70533f70-e19b-4768-9017-9029f53a6454 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:09:38.461Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:38.550Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T17:08:56.120689Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:39.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:09:40.064Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:40.950Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12fd04e5-c79d-4207-a34c-463c6260bf44 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:09:40.950Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12fd04e5-c79d-4207-a34c-463c6260bf44 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@77595ce9 2021-08-11T17:09:40.950Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12fd04e5-c79d-4207-a34c-463c6260bf44 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:09:40.950Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12fd04e5-c79d-4207-a34c-463c6260bf44 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:09:42.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:09:43.271Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:43.323Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:43.375Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:43.478Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:43.553Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:43.581Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:43.601Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b8c1653-3f5d-4dd5-959d-4e13acc87462 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:09:43.601Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b8c1653-3f5d-4dd5-959d-4e13acc87462 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7fc8449b 2021-08-11T17:09:43.601Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b8c1653-3f5d-4dd5-959d-4e13acc87462 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:09:43.601Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b8c1653-3f5d-4dd5-959d-4e13acc87462 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:09:43.605Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:43.658Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:43.683Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:43.760Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:43.863Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:43.885Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:43.965Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:44.087Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:44.168Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:44.290Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:44.572Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:44.693Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:44.975Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:45.095Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:45.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:09:45.779Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:45.899Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:09:46.606Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d9d5f908-6851-49c4-bfd2-3d02d6ced787 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:09:46.606Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d9d5f908-6851-49c4-bfd2-3d02d6ced787 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@72f1a9b6 2021-08-11T17:09:46.606Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d9d5f908-6851-49c4-bfd2-3d02d6ced787 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:09:46.606Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d9d5f908-6851-49c4-bfd2-3d02d6ced787 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:09:47.368Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=d9d5f908-6851-49c4-bfd2-3d02d6ced787 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T17:09:47.368Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=d9d5f908-6851-49c4-bfd2-3d02d6ced787 ErrorCategory=ERROR ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-08-11T17:09:47.684Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-08-11T17:09:48.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:09:48.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:09:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T17:09:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T17:09:49.292Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T17:09:49.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41e7994a-eddd-4256-9876-8b9314c4e36f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:09:49.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41e7994a-eddd-4256-9876-8b9314c4e36f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@82817f8 2021-08-11T17:09:49.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41e7994a-eddd-4256-9876-8b9314c4e36f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:09:49.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41e7994a-eddd-4256-9876-8b9314c4e36f PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:09:49.871Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=41e7994a-eddd-4256-9876-8b9314c4e36f ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T17:09:49.871Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=41e7994a-eddd-4256-9876-8b9314c4e36f ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-08-11T17:09:52.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=39fc61b1-0c65-4009-820d-b62c04da6976 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:09:52.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=39fc61b1-0c65-4009-820d-b62c04da6976 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@35abac00 2021-08-11T17:09:52.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=39fc61b1-0c65-4009-820d-b62c04da6976 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:09:52.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=39fc61b1-0c65-4009-820d-b62c04da6976 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:09:55.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=459f1d8b-1da3-4646-a459-d2d4ef5724a0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:09:55.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=459f1d8b-1da3-4646-a459-d2d4ef5724a0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@8facab2 2021-08-11T17:09:55.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=459f1d8b-1da3-4646-a459-d2d4ef5724a0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:09:55.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=459f1d8b-1da3-4646-a459-d2d4ef5724a0 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:10:00.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fc338ba7-781f-48dd-bb8b-97ea65e6493d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:10:00.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fc338ba7-781f-48dd-bb8b-97ea65e6493d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@77b96e02 2021-08-11T17:10:00.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fc338ba7-781f-48dd-bb8b-97ea65e6493d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:10:00.958Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fc338ba7-781f-48dd-bb8b-97ea65e6493d PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:10:02.677Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=fc338ba7-781f-48dd-bb8b-97ea65e6493d ErrorCategory=INFO ServiceName=BE Component CASSANDRA state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component CASSANDRA state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T17:10:02.678Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=fc338ba7-781f-48dd-bb8b-97ea65e6493d ErrorCategory=INFO ServiceName=BE Component CASSANDRA state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T17:10:03.051Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64d07aed-fec6-42f4-bb0c-042f3bbe1b13 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:10:03.051Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64d07aed-fec6-42f4-bb0c-042f3bbe1b13 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3fe630da 2021-08-11T17:10:03.051Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64d07aed-fec6-42f4-bb0c-042f3bbe1b13 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:10:03.051Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64d07aed-fec6-42f4-bb0c-042f3bbe1b13 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:10:03.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:10:03.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:10:04.258Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:10:04.709Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=81cfc123-b397-40df-a8f1-c3c150fae54b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:10:04.709Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=81cfc123-b397-40df-a8f1-c3c150fae54b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@57c5508a 2021-08-11T17:10:04.709Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=81cfc123-b397-40df-a8f1-c3c150fae54b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:10:04.709Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=81cfc123-b397-40df-a8f1-c3c150fae54b PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:10:07.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6661befd-e053-4a45-ab44-da709aee4bc1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:10:07.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6661befd-e053-4a45-ab44-da709aee4bc1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@9b4bf9a 2021-08-11T17:10:07.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6661befd-e053-4a45-ab44-da709aee4bc1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:10:07.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6661befd-e053-4a45-ab44-da709aee4bc1 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:10:11.050Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0d62e5f6-0ff4-4386-a9d7-4185ea5edda2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:10:11.050Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0d62e5f6-0ff4-4386-a9d7-4185ea5edda2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@354109d0 2021-08-11T17:10:11.050Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0d62e5f6-0ff4-4386-a9d7-4185ea5edda2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:10:11.051Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0d62e5f6-0ff4-4386-a9d7-4185ea5edda2 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:10:13.716Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=33f8cef3-ff48-4999-841a-a45bb8e64e46 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:10:13.716Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=33f8cef3-ff48-4999-841a-a45bb8e64e46 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3b2f8f42 2021-08-11T17:10:13.716Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=33f8cef3-ff48-4999-841a-a45bb8e64e46 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:10:13.716Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=33f8cef3-ff48-4999-841a-a45bb8e64e46 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:10:16.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb6f2d8a-f709-49d4-96f2-7f89ac47f336 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:10:16.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb6f2d8a-f709-49d4-96f2-7f89ac47f336 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4693c9ad 2021-08-11T17:10:16.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb6f2d8a-f709-49d4-96f2-7f89ac47f336 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:10:16.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bb6f2d8a-f709-49d4-96f2-7f89ac47f336 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:10:18.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:10:18.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:10:19.695Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=15aaf74e-bd46-4858-9bb9-b8b05b2de4ef PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:10:19.696Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=15aaf74e-bd46-4858-9bb9-b8b05b2de4ef PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6a77f68a 2021-08-11T17:10:19.696Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=15aaf74e-bd46-4858-9bb9-b8b05b2de4ef PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:10:19.696Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=15aaf74e-bd46-4858-9bb9-b8b05b2de4ef PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:10:22.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb1c0b35-7d00-48d7-903f-25230132e056 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:10:22.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb1c0b35-7d00-48d7-903f-25230132e056 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@57457d9d 2021-08-11T17:10:22.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb1c0b35-7d00-48d7-903f-25230132e056 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:10:22.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb1c0b35-7d00-48d7-903f-25230132e056 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:10:25.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0dfbb76c-c264-4587-b5d2-a9b8ec569166 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:10:25.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0dfbb76c-c264-4587-b5d2-a9b8ec569166 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1057bda8 2021-08-11T17:10:25.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0dfbb76c-c264-4587-b5d2-a9b8ec569166 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:10:25.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0dfbb76c-c264-4587-b5d2-a9b8ec569166 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:10:28.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=486709a8-8447-409a-b6ce-7bc24717f73a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:10:28.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=486709a8-8447-409a-b6ce-7bc24717f73a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@48d1c0ae 2021-08-11T17:10:28.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=486709a8-8447-409a-b6ce-7bc24717f73a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:10:28.858Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=486709a8-8447-409a-b6ce-7bc24717f73a PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:10:31.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8fabfea4-4bf7-4928-87a3-9891b5accc57 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:10:31.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8fabfea4-4bf7-4928-87a3-9891b5accc57 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1dc7ac98 2021-08-11T17:10:31.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8fabfea4-4bf7-4928-87a3-9891b5accc57 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:10:31.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8fabfea4-4bf7-4928-87a3-9891b5accc57 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:10:33.305Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:10:33.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:10:34.705Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc030bd1-b2bf-4f53-bcaf-8a4336d40b28 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:10:34.705Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc030bd1-b2bf-4f53-bcaf-8a4336d40b28 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@61128328 2021-08-11T17:10:34.705Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc030bd1-b2bf-4f53-bcaf-8a4336d40b28 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:10:34.705Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bc030bd1-b2bf-4f53-bcaf-8a4336d40b28 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:10:37.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3c87743-9fcf-419c-af59-60df7446126c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:10:37.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3c87743-9fcf-419c-af59-60df7446126c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4c05d85f 2021-08-11T17:10:37.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3c87743-9fcf-419c-af59-60df7446126c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:10:37.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3c87743-9fcf-419c-af59-60df7446126c PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:10:40.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f10da95d-4674-49da-b591-99b67eb640d4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:10:40.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f10da95d-4674-49da-b591-99b67eb640d4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@a17b5c1 2021-08-11T17:10:40.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f10da95d-4674-49da-b591-99b67eb640d4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:10:40.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f10da95d-4674-49da-b591-99b67eb640d4 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:10:43.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0d57ba86-fbbd-4a43-b9a3-abf5bd39409e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:10:43.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0d57ba86-fbbd-4a43-b9a3-abf5bd39409e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2a146b77 2021-08-11T17:10:43.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0d57ba86-fbbd-4a43-b9a3-abf5bd39409e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:10:43.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0d57ba86-fbbd-4a43-b9a3-abf5bd39409e PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:10:46.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d0639db9-6c19-43e5-b862-ad72c3176b72 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:10:46.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d0639db9-6c19-43e5-b862-ad72c3176b72 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5f6aa6de 2021-08-11T17:10:46.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d0639db9-6c19-43e5-b862-ad72c3176b72 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:10:46.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d0639db9-6c19-43e5-b862-ad72c3176b72 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:10:48.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:10:48.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:10:49.290Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T17:10:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T17:10:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T17:10:49.717Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06e1e2ee-13c5-492e-8fb6-e9e006f3762b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:10:49.717Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06e1e2ee-13c5-492e-8fb6-e9e006f3762b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3dcf4a4c 2021-08-11T17:10:49.717Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06e1e2ee-13c5-492e-8fb6-e9e006f3762b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:10:49.718Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=06e1e2ee-13c5-492e-8fb6-e9e006f3762b PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:10:52.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e038c840-f3cb-4e5e-8a4e-cd7a70471470 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:10:52.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e038c840-f3cb-4e5e-8a4e-cd7a70471470 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@a414989 2021-08-11T17:10:52.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e038c840-f3cb-4e5e-8a4e-cd7a70471470 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:10:52.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e038c840-f3cb-4e5e-8a4e-cd7a70471470 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:10:55.875Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce3f073e-f2c1-4c68-aa50-0589e67587a0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:10:55.875Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce3f073e-f2c1-4c68-aa50-0589e67587a0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@387d465 2021-08-11T17:10:55.875Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce3f073e-f2c1-4c68-aa50-0589e67587a0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:10:55.875Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ce3f073e-f2c1-4c68-aa50-0589e67587a0 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:10:58.652Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95234350-57a9-43a4-a211-83408a00b336 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:10:58.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95234350-57a9-43a4-a211-83408a00b336 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@9d1ce9d 2021-08-11T17:10:58.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95234350-57a9-43a4-a211-83408a00b336 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:10:58.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95234350-57a9-43a4-a211-83408a00b336 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:11:01.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ee70a7f-ef41-4d3d-888a-fae0c5a88976 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:11:01.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ee70a7f-ef41-4d3d-888a-fae0c5a88976 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4e0b68c1 2021-08-11T17:11:01.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ee70a7f-ef41-4d3d-888a-fae0c5a88976 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:11:01.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ee70a7f-ef41-4d3d-888a-fae0c5a88976 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:11:03.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:11:03.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:11:04.250Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:11:04.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa8b1139-c1b9-46f4-b8d2-17b84bd1b8ad PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:11:04.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa8b1139-c1b9-46f4-b8d2-17b84bd1b8ad PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4e067d88 2021-08-11T17:11:04.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa8b1139-c1b9-46f4-b8d2-17b84bd1b8ad PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:11:04.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fa8b1139-c1b9-46f4-b8d2-17b84bd1b8ad PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:11:07.609Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb646438-7029-4c04-a86a-5ca3dcf993bd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:11:07.609Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb646438-7029-4c04-a86a-5ca3dcf993bd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@739fbcd1 2021-08-11T17:11:07.609Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb646438-7029-4c04-a86a-5ca3dcf993bd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:11:07.609Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fb646438-7029-4c04-a86a-5ca3dcf993bd PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:11:10.687Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=20044dbb-280f-4d1a-957f-1910aa93bdc2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:11:10.687Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=20044dbb-280f-4d1a-957f-1910aa93bdc2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@743d3119 2021-08-11T17:11:10.687Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=20044dbb-280f-4d1a-957f-1910aa93bdc2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:11:10.687Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=20044dbb-280f-4d1a-957f-1910aa93bdc2 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:11:13.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3688e047-f3cb-4d84-aa01-4bd130c8c493 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:11:13.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3688e047-f3cb-4d84-aa01-4bd130c8c493 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@61e5123f 2021-08-11T17:11:13.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3688e047-f3cb-4d84-aa01-4bd130c8c493 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:11:13.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3688e047-f3cb-4d84-aa01-4bd130c8c493 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:11:16.688Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=538f48b1-ebb7-42e5-8c0a-b7b88b8b7048 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:11:16.688Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=538f48b1-ebb7-42e5-8c0a-b7b88b8b7048 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1cf2d9b6 2021-08-11T17:11:16.688Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=538f48b1-ebb7-42e5-8c0a-b7b88b8b7048 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:11:16.688Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=538f48b1-ebb7-42e5-8c0a-b7b88b8b7048 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:11:18.310Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:11:18.320Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:11:19.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=319d3654-12af-43d3-b311-22761584e4dd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:11:19.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=319d3654-12af-43d3-b311-22761584e4dd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@120597cd 2021-08-11T17:11:19.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=319d3654-12af-43d3-b311-22761584e4dd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:11:19.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=319d3654-12af-43d3-b311-22761584e4dd PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:11:22.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f47bd99d-e79d-44bb-9ae7-85e1070a3744 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:11:22.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f47bd99d-e79d-44bb-9ae7-85e1070a3744 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@f2dc96d 2021-08-11T17:11:22.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f47bd99d-e79d-44bb-9ae7-85e1070a3744 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:11:22.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f47bd99d-e79d-44bb-9ae7-85e1070a3744 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:11:25.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d992b28-a217-4e9e-b025-7d48294a9bf4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:11:25.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d992b28-a217-4e9e-b025-7d48294a9bf4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4690565d 2021-08-11T17:11:25.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d992b28-a217-4e9e-b025-7d48294a9bf4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:11:25.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d992b28-a217-4e9e-b025-7d48294a9bf4 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:11:28.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=54a8988f-4555-4ae9-957b-ac1c84b44eda PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:11:28.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=54a8988f-4555-4ae9-957b-ac1c84b44eda PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4c251850 2021-08-11T17:11:28.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=54a8988f-4555-4ae9-957b-ac1c84b44eda PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:11:28.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=54a8988f-4555-4ae9-957b-ac1c84b44eda PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:11:31.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f120fa4f-c2de-4ded-bd5a-141f716bd1c7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:11:31.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f120fa4f-c2de-4ded-bd5a-141f716bd1c7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@722055e6 2021-08-11T17:11:31.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f120fa4f-c2de-4ded-bd5a-141f716bd1c7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:11:31.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f120fa4f-c2de-4ded-bd5a-141f716bd1c7 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:11:33.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:11:33.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:11:34.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d1d3e23c-058d-43de-a6d7-3a2abb3a9cdb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:11:34.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d1d3e23c-058d-43de-a6d7-3a2abb3a9cdb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@13278c6e 2021-08-11T17:11:34.625Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d1d3e23c-058d-43de-a6d7-3a2abb3a9cdb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:11:34.626Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d1d3e23c-058d-43de-a6d7-3a2abb3a9cdb PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:11:37.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2939c4ab-9284-408d-a36f-4402a0cc90e8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:11:37.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2939c4ab-9284-408d-a36f-4402a0cc90e8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@574102e9 2021-08-11T17:11:37.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2939c4ab-9284-408d-a36f-4402a0cc90e8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:11:37.672Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2939c4ab-9284-408d-a36f-4402a0cc90e8 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:11:40.852Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95877632-60e5-4742-a0ce-299ee8ea38a4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:11:40.852Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95877632-60e5-4742-a0ce-299ee8ea38a4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@33c267c9 2021-08-11T17:11:40.852Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95877632-60e5-4742-a0ce-299ee8ea38a4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:11:40.852Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95877632-60e5-4742-a0ce-299ee8ea38a4 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:11:43.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10d87b6f-d68c-4fdf-b5b8-8d2533951154 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:11:43.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10d87b6f-d68c-4fdf-b5b8-8d2533951154 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@43bd9b32 2021-08-11T17:11:43.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10d87b6f-d68c-4fdf-b5b8-8d2533951154 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:11:43.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10d87b6f-d68c-4fdf-b5b8-8d2533951154 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:11:46.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4783081a-9070-4f6c-bc74-d1f47978c378 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:11:46.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4783081a-9070-4f6c-bc74-d1f47978c378 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6c562bd7 2021-08-11T17:11:46.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4783081a-9070-4f6c-bc74-d1f47978c378 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:11:46.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4783081a-9070-4f6c-bc74-d1f47978c378 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:11:48.309Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:11:48.318Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:11:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T17:11:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T17:11:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T17:11:50.050Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3350d11-39e9-470e-be06-9924afd6797f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:11:50.051Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3350d11-39e9-470e-be06-9924afd6797f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@33df2afe 2021-08-11T17:11:50.051Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3350d11-39e9-470e-be06-9924afd6797f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:11:50.051Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b3350d11-39e9-470e-be06-9924afd6797f PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:11:53.167Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bfe0187f-394b-412e-bc01-ea5b090a16d9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:11:53.167Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bfe0187f-394b-412e-bc01-ea5b090a16d9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1383ff63 2021-08-11T17:11:53.167Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bfe0187f-394b-412e-bc01-ea5b090a16d9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:11:53.167Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bfe0187f-394b-412e-bc01-ea5b090a16d9 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:11:55.871Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=898074ee-eb91-4b6f-8bf8-c569cf755c92 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:11:55.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=898074ee-eb91-4b6f-8bf8-c569cf755c92 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3c48822a 2021-08-11T17:11:55.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=898074ee-eb91-4b6f-8bf8-c569cf755c92 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:11:55.872Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=898074ee-eb91-4b6f-8bf8-c569cf755c92 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:11:58.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90bd33ca-3957-4851-9672-c75f317b8596 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:11:58.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90bd33ca-3957-4851-9672-c75f317b8596 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@611d1ee7 2021-08-11T17:11:58.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90bd33ca-3957-4851-9672-c75f317b8596 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:11:58.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90bd33ca-3957-4851-9672-c75f317b8596 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:12:01.586Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e26b0055-7684-42be-b622-118067481d43 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:12:01.586Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e26b0055-7684-42be-b622-118067481d43 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2b5f40e0 2021-08-11T17:12:01.586Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e26b0055-7684-42be-b622-118067481d43 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:12:01.586Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e26b0055-7684-42be-b622-118067481d43 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:12:03.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:12:03.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:12:04.257Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:12:04.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4dbd3487-f62d-4a41-9508-e16feb719976 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:12:04.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4dbd3487-f62d-4a41-9508-e16feb719976 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@27abca0f 2021-08-11T17:12:04.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4dbd3487-f62d-4a41-9508-e16feb719976 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:12:04.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4dbd3487-f62d-4a41-9508-e16feb719976 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:12:07.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f889ba6-2429-4fd9-a2b8-aabba0bed099 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:12:07.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f889ba6-2429-4fd9-a2b8-aabba0bed099 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5f8a4502 2021-08-11T17:12:07.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f889ba6-2429-4fd9-a2b8-aabba0bed099 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:12:07.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f889ba6-2429-4fd9-a2b8-aabba0bed099 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:12:10.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e366227-d4f4-4d8c-940e-f7c600ac2a6a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:12:10.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e366227-d4f4-4d8c-940e-f7c600ac2a6a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@56e98232 2021-08-11T17:12:10.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e366227-d4f4-4d8c-940e-f7c600ac2a6a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:12:10.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e366227-d4f4-4d8c-940e-f7c600ac2a6a PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:12:13.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cd021281-a5cf-48df-bd2d-5fe3a34925d2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:12:13.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cd021281-a5cf-48df-bd2d-5fe3a34925d2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7e0804f 2021-08-11T17:12:13.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cd021281-a5cf-48df-bd2d-5fe3a34925d2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:12:13.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cd021281-a5cf-48df-bd2d-5fe3a34925d2 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:12:16.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e3480358-2e03-434b-bdd6-118da6e99bc7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:12:16.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e3480358-2e03-434b-bdd6-118da6e99bc7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7e62d90e 2021-08-11T17:12:16.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e3480358-2e03-434b-bdd6-118da6e99bc7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:12:16.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e3480358-2e03-434b-bdd6-118da6e99bc7 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:12:18.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:12:18.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:12:19.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8ebb69d9-7966-4657-8751-5455f6c7e6eb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:12:19.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8ebb69d9-7966-4657-8751-5455f6c7e6eb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@73cd3195 2021-08-11T17:12:19.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8ebb69d9-7966-4657-8751-5455f6c7e6eb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:12:19.677Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8ebb69d9-7966-4657-8751-5455f6c7e6eb PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:12:22.660Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0eb4d90a-0cd4-4135-81db-a450ec7f4bfb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:12:22.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0eb4d90a-0cd4-4135-81db-a450ec7f4bfb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@69afc5cc 2021-08-11T17:12:22.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0eb4d90a-0cd4-4135-81db-a450ec7f4bfb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:12:22.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0eb4d90a-0cd4-4135-81db-a450ec7f4bfb PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:12:25.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=43f35ef7-a321-4bef-ae25-279e279dfa82 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:12:25.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=43f35ef7-a321-4bef-ae25-279e279dfa82 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6aced24e 2021-08-11T17:12:25.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=43f35ef7-a321-4bef-ae25-279e279dfa82 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:12:25.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=43f35ef7-a321-4bef-ae25-279e279dfa82 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:12:28.851Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=81633499-fa68-4e99-84a8-3a6075c803ca PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:12:28.851Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=81633499-fa68-4e99-84a8-3a6075c803ca PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@28cccb0 2021-08-11T17:12:28.851Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=81633499-fa68-4e99-84a8-3a6075c803ca PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:12:28.851Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=81633499-fa68-4e99-84a8-3a6075c803ca PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:12:31.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3297f083-43ab-41f8-91ac-cd0cded5b425 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:12:31.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3297f083-43ab-41f8-91ac-cd0cded5b425 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@185714ed 2021-08-11T17:12:31.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3297f083-43ab-41f8-91ac-cd0cded5b425 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:12:31.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3297f083-43ab-41f8-91ac-cd0cded5b425 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:12:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:12:33.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:12:34.599Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e817b14f-79a5-4da2-a720-b5136f199365 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:12:34.599Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e817b14f-79a5-4da2-a720-b5136f199365 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@fc1238e 2021-08-11T17:12:34.599Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e817b14f-79a5-4da2-a720-b5136f199365 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:12:34.599Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e817b14f-79a5-4da2-a720-b5136f199365 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:12:37.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=92a4e6b0-1b8b-4aca-adc7-047980178913 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:12:37.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=92a4e6b0-1b8b-4aca-adc7-047980178913 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1301cc9a 2021-08-11T17:12:37.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=92a4e6b0-1b8b-4aca-adc7-047980178913 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:12:37.757Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=92a4e6b0-1b8b-4aca-adc7-047980178913 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:12:40.590Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=996f1c9e-3e52-4fb2-9e2c-27060542d8f7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:12:40.590Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=996f1c9e-3e52-4fb2-9e2c-27060542d8f7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@778c6b06 2021-08-11T17:12:40.590Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=996f1c9e-3e52-4fb2-9e2c-27060542d8f7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:12:40.590Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=996f1c9e-3e52-4fb2-9e2c-27060542d8f7 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:12:43.613Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7b58994-bb47-48fe-a94b-4fff7647e676 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:12:43.613Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7b58994-bb47-48fe-a94b-4fff7647e676 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@575ce966 2021-08-11T17:12:43.613Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7b58994-bb47-48fe-a94b-4fff7647e676 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:12:43.613Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b7b58994-bb47-48fe-a94b-4fff7647e676 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:12:46.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a2144fa-d390-4d31-8ecc-857cdaa1f69a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:12:46.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a2144fa-d390-4d31-8ecc-857cdaa1f69a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@51076d50 2021-08-11T17:12:46.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a2144fa-d390-4d31-8ecc-857cdaa1f69a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:12:46.611Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1a2144fa-d390-4d31-8ecc-857cdaa1f69a PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:12:48.305Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:12:48.310Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:12:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T17:12:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T17:12:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T17:12:50.450Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2deece51-2d69-4cae-bd11-a12f5fbff368 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:12:50.450Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2deece51-2d69-4cae-bd11-a12f5fbff368 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4c7f0ef6 2021-08-11T17:12:50.450Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2deece51-2d69-4cae-bd11-a12f5fbff368 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:12:50.450Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2deece51-2d69-4cae-bd11-a12f5fbff368 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:12:53.346Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fc34e84c-9ed1-4ed5-9d9a-1778817dc238 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:12:53.346Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fc34e84c-9ed1-4ed5-9d9a-1778817dc238 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@58bf9824 2021-08-11T17:12:53.346Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fc34e84c-9ed1-4ed5-9d9a-1778817dc238 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:12:53.346Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fc34e84c-9ed1-4ed5-9d9a-1778817dc238 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:12:56.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0eb36c2c-d62f-48ee-9ecb-84bb72bb3868 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:12:56.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0eb36c2c-d62f-48ee-9ecb-84bb72bb3868 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@68184d8c 2021-08-11T17:12:56.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0eb36c2c-d62f-48ee-9ecb-84bb72bb3868 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:12:56.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0eb36c2c-d62f-48ee-9ecb-84bb72bb3868 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:13:00.060Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f023654d-e208-47bd-b8f3-2ebf1486482b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:13:00.060Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f023654d-e208-47bd-b8f3-2ebf1486482b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@29fc032 2021-08-11T17:13:00.060Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f023654d-e208-47bd-b8f3-2ebf1486482b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:13:00.060Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f023654d-e208-47bd-b8f3-2ebf1486482b PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:13:02.553Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=828ec280-5078-4cc9-9ebf-b11eb07de6a4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:13:02.553Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=828ec280-5078-4cc9-9ebf-b11eb07de6a4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@36a36629 2021-08-11T17:13:02.553Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=828ec280-5078-4cc9-9ebf-b11eb07de6a4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:13:02.553Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=828ec280-5078-4cc9-9ebf-b11eb07de6a4 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:13:03.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:13:03.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:13:04.266Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:13:04.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b1b59f4-6580-4799-a094-e955205a7da7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:13:04.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b1b59f4-6580-4799-a094-e955205a7da7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@12f9b661 2021-08-11T17:13:04.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b1b59f4-6580-4799-a094-e955205a7da7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:13:04.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7b1b59f4-6580-4799-a094-e955205a7da7 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:13:07.611Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f60a74c-09ae-452d-b448-cd420610b799 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:13:07.611Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f60a74c-09ae-452d-b448-cd420610b799 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3827fd9f 2021-08-11T17:13:07.611Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f60a74c-09ae-452d-b448-cd420610b799 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:13:07.611Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0f60a74c-09ae-452d-b448-cd420610b799 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:13:10.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d4f606ea-8f43-48af-87c5-01df5e8fb723 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:13:10.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d4f606ea-8f43-48af-87c5-01df5e8fb723 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@44c534f9 2021-08-11T17:13:10.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d4f606ea-8f43-48af-87c5-01df5e8fb723 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:13:10.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d4f606ea-8f43-48af-87c5-01df5e8fb723 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:13:13.602Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3acc4a96-c540-4f7c-b282-3f978893193f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:13:13.602Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3acc4a96-c540-4f7c-b282-3f978893193f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@69fa8066 2021-08-11T17:13:13.602Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3acc4a96-c540-4f7c-b282-3f978893193f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:13:13.603Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3acc4a96-c540-4f7c-b282-3f978893193f PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:13:16.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d311693-dc9b-4729-beea-c748b4f08906 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:13:16.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d311693-dc9b-4729-beea-c748b4f08906 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@36ef6986 2021-08-11T17:13:16.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d311693-dc9b-4729-beea-c748b4f08906 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:13:16.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d311693-dc9b-4729-beea-c748b4f08906 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:13:18.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:13:18.320Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:13:19.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aefa7d5f-175d-4e48-9bee-f270d97a631d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:13:19.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aefa7d5f-175d-4e48-9bee-f270d97a631d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3123b1c1 2021-08-11T17:13:19.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aefa7d5f-175d-4e48-9bee-f270d97a631d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:13:19.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=aefa7d5f-175d-4e48-9bee-f270d97a631d PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:13:22.617Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b8040e9-785a-46ee-974c-3e2e0fbf47b1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:13:22.617Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b8040e9-785a-46ee-974c-3e2e0fbf47b1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5c2c39e0 2021-08-11T17:13:22.617Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b8040e9-785a-46ee-974c-3e2e0fbf47b1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:13:22.617Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5b8040e9-785a-46ee-974c-3e2e0fbf47b1 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:13:25.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b64f66fc-bd07-48ac-9e71-ff4677a51bf4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:13:25.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b64f66fc-bd07-48ac-9e71-ff4677a51bf4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@61e86bb3 2021-08-11T17:13:25.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b64f66fc-bd07-48ac-9e71-ff4677a51bf4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:13:25.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b64f66fc-bd07-48ac-9e71-ff4677a51bf4 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:13:28.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96f2efa1-908e-4382-a2a4-10b3194eada5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:13:28.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96f2efa1-908e-4382-a2a4-10b3194eada5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2c868d51 2021-08-11T17:13:28.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96f2efa1-908e-4382-a2a4-10b3194eada5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:13:28.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=96f2efa1-908e-4382-a2a4-10b3194eada5 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:13:31.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e3090c38-096e-4e1e-8a55-d1dc3f80e75f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:13:31.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e3090c38-096e-4e1e-8a55-d1dc3f80e75f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@879cb9 2021-08-11T17:13:31.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e3090c38-096e-4e1e-8a55-d1dc3f80e75f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:13:31.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e3090c38-096e-4e1e-8a55-d1dc3f80e75f PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:13:33.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:13:33.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:13:34.612Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=80cbccf0-ff59-40bd-9bb1-dc92f06765e2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:13:34.612Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=80cbccf0-ff59-40bd-9bb1-dc92f06765e2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3dd95670 2021-08-11T17:13:34.612Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=80cbccf0-ff59-40bd-9bb1-dc92f06765e2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:13:34.612Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=80cbccf0-ff59-40bd-9bb1-dc92f06765e2 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:13:37.717Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f67981d5-978a-4fba-94cb-3a8e1c98a4a4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:13:37.717Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f67981d5-978a-4fba-94cb-3a8e1c98a4a4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1fce178b 2021-08-11T17:13:37.717Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f67981d5-978a-4fba-94cb-3a8e1c98a4a4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:13:37.717Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f67981d5-978a-4fba-94cb-3a8e1c98a4a4 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:13:40.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8769bf21-0b80-4383-8dd5-95a914f99096 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:13:40.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8769bf21-0b80-4383-8dd5-95a914f99096 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@73aed26 2021-08-11T17:13:40.864Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8769bf21-0b80-4383-8dd5-95a914f99096 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:13:40.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8769bf21-0b80-4383-8dd5-95a914f99096 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:13:43.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66620dbc-2430-4340-ad51-78a1062f6dee PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:13:43.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66620dbc-2430-4340-ad51-78a1062f6dee PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3be4cb2c 2021-08-11T17:13:43.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66620dbc-2430-4340-ad51-78a1062f6dee PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:13:43.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66620dbc-2430-4340-ad51-78a1062f6dee PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:13:46.609Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=343cffcd-e5ce-4d23-9b75-f644c770eb5f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:13:46.609Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=343cffcd-e5ce-4d23-9b75-f644c770eb5f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4c7970f6 2021-08-11T17:13:46.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=343cffcd-e5ce-4d23-9b75-f644c770eb5f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:13:46.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=343cffcd-e5ce-4d23-9b75-f644c770eb5f PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:13:48.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:13:48.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:13:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T17:13:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T17:13:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T17:13:49.948Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd3904cc-1df9-413b-a2cf-65cb2ebcb3c3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:13:49.948Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd3904cc-1df9-413b-a2cf-65cb2ebcb3c3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@23e454bd 2021-08-11T17:13:49.949Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd3904cc-1df9-413b-a2cf-65cb2ebcb3c3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:13:49.949Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bd3904cc-1df9-413b-a2cf-65cb2ebcb3c3 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:13:53.158Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5510eb8a-554b-4664-9a59-072a851ef72a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:13:53.158Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5510eb8a-554b-4664-9a59-072a851ef72a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@73e7de3a 2021-08-11T17:13:53.158Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5510eb8a-554b-4664-9a59-072a851ef72a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:13:53.158Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5510eb8a-554b-4664-9a59-072a851ef72a PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:13:55.652Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db35331a-2d60-47cc-9110-a6a2c1a2a9b3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:13:55.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db35331a-2d60-47cc-9110-a6a2c1a2a9b3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6e47c6c1 2021-08-11T17:13:55.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db35331a-2d60-47cc-9110-a6a2c1a2a9b3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:13:55.653Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db35331a-2d60-47cc-9110-a6a2c1a2a9b3 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:13:58.603Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1c7d341f-b251-46a7-b5b5-5dbd7ca5cc61 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:13:58.603Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1c7d341f-b251-46a7-b5b5-5dbd7ca5cc61 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@69bef53f 2021-08-11T17:13:58.603Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1c7d341f-b251-46a7-b5b5-5dbd7ca5cc61 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:13:58.603Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1c7d341f-b251-46a7-b5b5-5dbd7ca5cc61 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:14:01.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4ba06bc-1047-42c5-987b-41770a82973c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:14:01.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4ba06bc-1047-42c5-987b-41770a82973c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@68fe0509 2021-08-11T17:14:01.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4ba06bc-1047-42c5-987b-41770a82973c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:14:01.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e4ba06bc-1047-42c5-987b-41770a82973c PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:14:03.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:14:03.310Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:14:04.257Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:14:04.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c33c98df-bdb3-4a58-a7c6-7483e1d8e4de PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:14:04.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c33c98df-bdb3-4a58-a7c6-7483e1d8e4de PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@25a51022 2021-08-11T17:14:04.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c33c98df-bdb3-4a58-a7c6-7483e1d8e4de PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:14:04.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c33c98df-bdb3-4a58-a7c6-7483e1d8e4de PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:14:07.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3e478fcf-c879-4044-9789-2acd9a2c3b1c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:14:07.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3e478fcf-c879-4044-9789-2acd9a2c3b1c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7584ab99 2021-08-11T17:14:07.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3e478fcf-c879-4044-9789-2acd9a2c3b1c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:14:07.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3e478fcf-c879-4044-9789-2acd9a2c3b1c PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:14:10.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71fd14c1-875a-4038-8e91-12a23a28a253 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:14:10.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71fd14c1-875a-4038-8e91-12a23a28a253 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6b0dbf1b 2021-08-11T17:14:10.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71fd14c1-875a-4038-8e91-12a23a28a253 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:14:10.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=71fd14c1-875a-4038-8e91-12a23a28a253 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:14:13.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29957111-dc3f-45b7-981a-a5f689212f50 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:14:13.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29957111-dc3f-45b7-981a-a5f689212f50 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7343393e 2021-08-11T17:14:13.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29957111-dc3f-45b7-981a-a5f689212f50 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:14:13.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=29957111-dc3f-45b7-981a-a5f689212f50 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:14:16.605Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f293a3f3-bd45-454a-9b5a-c016e4487d44 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:14:16.606Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f293a3f3-bd45-454a-9b5a-c016e4487d44 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7e6a1374 2021-08-11T17:14:16.606Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f293a3f3-bd45-454a-9b5a-c016e4487d44 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:14:16.606Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f293a3f3-bd45-454a-9b5a-c016e4487d44 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:14:18.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:14:18.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:14:19.611Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7ab2187e-f7c7-4578-a30b-819bf921c245 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:14:19.611Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7ab2187e-f7c7-4578-a30b-819bf921c245 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2ee2a743 2021-08-11T17:14:19.611Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7ab2187e-f7c7-4578-a30b-819bf921c245 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:14:19.611Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7ab2187e-f7c7-4578-a30b-819bf921c245 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:14:22.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f50c06c6-4414-4c84-9a20-2b8303eb321b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:14:22.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f50c06c6-4414-4c84-9a20-2b8303eb321b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@289130e3 2021-08-11T17:14:22.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f50c06c6-4414-4c84-9a20-2b8303eb321b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:14:22.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f50c06c6-4414-4c84-9a20-2b8303eb321b PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:14:25.598Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4d073d0-d22a-49fc-aac2-49984f3013a5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:14:25.598Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4d073d0-d22a-49fc-aac2-49984f3013a5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@42716606 2021-08-11T17:14:25.598Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4d073d0-d22a-49fc-aac2-49984f3013a5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:14:25.599Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4d073d0-d22a-49fc-aac2-49984f3013a5 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:14:28.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8f64af84-020c-42e5-834d-568eca21a2cb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:14:28.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8f64af84-020c-42e5-834d-568eca21a2cb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6cae3c8f 2021-08-11T17:14:28.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8f64af84-020c-42e5-834d-568eca21a2cb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:14:28.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8f64af84-020c-42e5-834d-568eca21a2cb PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:14:31.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1c5a0008-fe94-480e-a5f6-5c832f0aca65 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:14:31.623Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1c5a0008-fe94-480e-a5f6-5c832f0aca65 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@574d46e6 2021-08-11T17:14:31.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1c5a0008-fe94-480e-a5f6-5c832f0aca65 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:14:31.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1c5a0008-fe94-480e-a5f6-5c832f0aca65 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:14:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:14:33.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:14:34.658Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72ac145f-101d-4c41-a102-4136c219972e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:14:34.658Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72ac145f-101d-4c41-a102-4136c219972e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@767a62c4 2021-08-11T17:14:34.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72ac145f-101d-4c41-a102-4136c219972e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:14:34.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72ac145f-101d-4c41-a102-4136c219972e PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:14:37.596Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2023d436-1504-4ecb-96b1-894a4bac4b7d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:14:37.596Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2023d436-1504-4ecb-96b1-894a4bac4b7d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3f726b5b 2021-08-11T17:14:37.596Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2023d436-1504-4ecb-96b1-894a4bac4b7d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:14:37.596Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2023d436-1504-4ecb-96b1-894a4bac4b7d PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:14:40.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9880248a-9fa9-4fd8-882e-cf3e31a0fc61 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:14:40.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9880248a-9fa9-4fd8-882e-cf3e31a0fc61 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@22a5727e 2021-08-11T17:14:40.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9880248a-9fa9-4fd8-882e-cf3e31a0fc61 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:14:40.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9880248a-9fa9-4fd8-882e-cf3e31a0fc61 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:14:43.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea9d35be-83d4-4fd0-9c13-637be62170a3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:14:43.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea9d35be-83d4-4fd0-9c13-637be62170a3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1f627aba 2021-08-11T17:14:43.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea9d35be-83d4-4fd0-9c13-637be62170a3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:14:43.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ea9d35be-83d4-4fd0-9c13-637be62170a3 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:14:46.590Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e81597c-9865-42ce-a9d9-75424fad7d65 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:14:46.590Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e81597c-9865-42ce-a9d9-75424fad7d65 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3a2b47d0 2021-08-11T17:14:46.590Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e81597c-9865-42ce-a9d9-75424fad7d65 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:14:46.590Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5e81597c-9865-42ce-a9d9-75424fad7d65 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:14:48.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:14:48.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:14:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T17:14:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T17:14:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T17:14:50.153Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2270fc90-ccc1-4482-86bd-6671b4ac4e86 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:14:50.153Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2270fc90-ccc1-4482-86bd-6671b4ac4e86 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@162fcb7d 2021-08-11T17:14:50.153Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2270fc90-ccc1-4482-86bd-6671b4ac4e86 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:14:50.153Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2270fc90-ccc1-4482-86bd-6671b4ac4e86 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:14:53.547Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e6b023b-dd29-4895-8c13-2d35a995aa16 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:14:53.547Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e6b023b-dd29-4895-8c13-2d35a995aa16 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1f106d44 2021-08-11T17:14:53.547Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e6b023b-dd29-4895-8c13-2d35a995aa16 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:14:53.547Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e6b023b-dd29-4895-8c13-2d35a995aa16 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:14:56.452Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9e30a85-dd49-4b30-87e6-8ab6ce08c764 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:14:56.452Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9e30a85-dd49-4b30-87e6-8ab6ce08c764 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@132b63b6 2021-08-11T17:14:56.452Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9e30a85-dd49-4b30-87e6-8ab6ce08c764 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:14:56.452Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9e30a85-dd49-4b30-87e6-8ab6ce08c764 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:14:59.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90c76d5d-850c-47da-97db-7c0cfa4b8719 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:14:59.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90c76d5d-850c-47da-97db-7c0cfa4b8719 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@73deb3c4 2021-08-11T17:14:59.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90c76d5d-850c-47da-97db-7c0cfa4b8719 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:14:59.655Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=90c76d5d-850c-47da-97db-7c0cfa4b8719 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:15:01.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f92c560c-6776-4f6b-81ef-6e8264cdf040 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:15:01.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f92c560c-6776-4f6b-81ef-6e8264cdf040 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1b42fae1 2021-08-11T17:15:01.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f92c560c-6776-4f6b-81ef-6e8264cdf040 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:15:01.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f92c560c-6776-4f6b-81ef-6e8264cdf040 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:15:03.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:15:03.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:15:04.252Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:15:04.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f084d4a-2bf0-463a-8652-b314d6438c82 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:15:04.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f084d4a-2bf0-463a-8652-b314d6438c82 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@724ce80 2021-08-11T17:15:04.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f084d4a-2bf0-463a-8652-b314d6438c82 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:15:04.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6f084d4a-2bf0-463a-8652-b314d6438c82 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:15:07.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e306bb5f-a764-43b7-bf74-0f537342472a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:15:07.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e306bb5f-a764-43b7-bf74-0f537342472a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4f5a2b01 2021-08-11T17:15:07.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e306bb5f-a764-43b7-bf74-0f537342472a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:15:07.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e306bb5f-a764-43b7-bf74-0f537342472a PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:15:10.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=be9b1ce5-c1ca-415e-8701-c6fa0efe1de8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:15:10.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=be9b1ce5-c1ca-415e-8701-c6fa0efe1de8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@491cf1bd 2021-08-11T17:15:10.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=be9b1ce5-c1ca-415e-8701-c6fa0efe1de8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:15:10.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=be9b1ce5-c1ca-415e-8701-c6fa0efe1de8 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:15:14.166Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef401fa9-57de-4c42-bb87-3a717f1151b0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:15:14.166Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef401fa9-57de-4c42-bb87-3a717f1151b0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@45859f8d 2021-08-11T17:15:14.166Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef401fa9-57de-4c42-bb87-3a717f1151b0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:15:14.166Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ef401fa9-57de-4c42-bb87-3a717f1151b0 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:15:16.613Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c99551ba-988d-42f3-9244-ff049462a2fb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:15:16.613Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c99551ba-988d-42f3-9244-ff049462a2fb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@64f40973 2021-08-11T17:15:16.613Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c99551ba-988d-42f3-9244-ff049462a2fb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:15:16.613Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c99551ba-988d-42f3-9244-ff049462a2fb PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:15:18.310Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:15:18.319Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:15:19.627Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24b9d4e9-65fb-4d0b-849b-862b7f84a3d5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:15:19.627Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24b9d4e9-65fb-4d0b-849b-862b7f84a3d5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7a6824cf 2021-08-11T17:15:19.627Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24b9d4e9-65fb-4d0b-849b-862b7f84a3d5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:15:19.627Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24b9d4e9-65fb-4d0b-849b-862b7f84a3d5 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:15:22.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=009627d5-0a82-4410-9b5e-1fb9585b8c04 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:15:22.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=009627d5-0a82-4410-9b5e-1fb9585b8c04 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5457b326 2021-08-11T17:15:22.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=009627d5-0a82-4410-9b5e-1fb9585b8c04 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:15:22.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=009627d5-0a82-4410-9b5e-1fb9585b8c04 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:15:25.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14504ddd-fffc-489d-8609-482f23eca740 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:15:25.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14504ddd-fffc-489d-8609-482f23eca740 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3d4737d6 2021-08-11T17:15:25.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14504ddd-fffc-489d-8609-482f23eca740 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:15:25.661Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14504ddd-fffc-489d-8609-482f23eca740 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:15:28.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=209a7f5f-39dd-4e86-a324-dc546e7b3d4e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:15:28.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=209a7f5f-39dd-4e86-a324-dc546e7b3d4e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@793c1e9c 2021-08-11T17:15:28.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=209a7f5f-39dd-4e86-a324-dc546e7b3d4e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:15:28.857Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=209a7f5f-39dd-4e86-a324-dc546e7b3d4e PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:15:31.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee8d88e2-7ef7-4b01-9f56-239a5a7670a6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:15:31.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee8d88e2-7ef7-4b01-9f56-239a5a7670a6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@42c91328 2021-08-11T17:15:31.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee8d88e2-7ef7-4b01-9f56-239a5a7670a6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:15:31.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ee8d88e2-7ef7-4b01-9f56-239a5a7670a6 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:15:33.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:15:33.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:15:34.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a33e626-5203-446e-ab56-2256016d70c4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:15:34.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a33e626-5203-446e-ab56-2256016d70c4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2bee56d4 2021-08-11T17:15:34.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a33e626-5203-446e-ab56-2256016d70c4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:15:34.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a33e626-5203-446e-ab56-2256016d70c4 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:15:37.750Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=adab6226-5842-4bc8-ad21-f60595eecfcf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:15:37.750Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=adab6226-5842-4bc8-ad21-f60595eecfcf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2f133df8 2021-08-11T17:15:37.750Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=adab6226-5842-4bc8-ad21-f60595eecfcf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:15:37.750Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=adab6226-5842-4bc8-ad21-f60595eecfcf PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:15:40.851Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f6679327-1476-4135-9332-51cebe001fb9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:15:40.851Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f6679327-1476-4135-9332-51cebe001fb9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@36c21e78 2021-08-11T17:15:40.851Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f6679327-1476-4135-9332-51cebe001fb9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:15:40.852Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f6679327-1476-4135-9332-51cebe001fb9 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:15:43.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=734ea7f0-7b0c-455c-a7d4-4ac98016039b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:15:43.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=734ea7f0-7b0c-455c-a7d4-4ac98016039b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3c980bdb 2021-08-11T17:15:43.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=734ea7f0-7b0c-455c-a7d4-4ac98016039b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:15:43.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=734ea7f0-7b0c-455c-a7d4-4ac98016039b PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:15:46.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c9bcdc82-0178-4b61-95af-253cf87dfe1f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:15:46.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c9bcdc82-0178-4b61-95af-253cf87dfe1f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6519b26a 2021-08-11T17:15:46.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c9bcdc82-0178-4b61-95af-253cf87dfe1f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:15:46.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c9bcdc82-0178-4b61-95af-253cf87dfe1f PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:15:48.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:15:48.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:15:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T17:15:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T17:15:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T17:15:49.945Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=940ae48b-c8f3-4b8f-b6ed-dafd48103afd PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:15:49.946Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=940ae48b-c8f3-4b8f-b6ed-dafd48103afd PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@53d3bfcc 2021-08-11T17:15:49.946Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=940ae48b-c8f3-4b8f-b6ed-dafd48103afd PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:15:49.946Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=940ae48b-c8f3-4b8f-b6ed-dafd48103afd PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:15:53.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b56627e-a273-4236-bb26-9a233dda1907 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:15:53.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b56627e-a273-4236-bb26-9a233dda1907 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@75445a10 2021-08-11T17:15:53.057Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b56627e-a273-4236-bb26-9a233dda1907 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:15:53.058Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b56627e-a273-4236-bb26-9a233dda1907 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:15:55.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10073b03-44fc-4b68-a0f2-e3bf5720d79f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:15:55.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10073b03-44fc-4b68-a0f2-e3bf5720d79f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@9fe51b0 2021-08-11T17:15:55.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10073b03-44fc-4b68-a0f2-e3bf5720d79f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:15:55.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10073b03-44fc-4b68-a0f2-e3bf5720d79f PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:15:58.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b10d670d-ce7c-458c-a1fa-83437f82fe47 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:15:58.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b10d670d-ce7c-458c-a1fa-83437f82fe47 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@21dc2807 2021-08-11T17:15:58.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b10d670d-ce7c-458c-a1fa-83437f82fe47 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:15:58.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b10d670d-ce7c-458c-a1fa-83437f82fe47 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:16:01.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cdcd2869-1fe0-465b-99e6-0c5233cfcda8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:16:01.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cdcd2869-1fe0-465b-99e6-0c5233cfcda8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6f3fde2f 2021-08-11T17:16:01.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cdcd2869-1fe0-465b-99e6-0c5233cfcda8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:16:01.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cdcd2869-1fe0-465b-99e6-0c5233cfcda8 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:16:03.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:16:03.311Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:16:04.248Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:16:04.654Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=216d65cb-29e1-4120-8836-829870c1351b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:16:04.654Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=216d65cb-29e1-4120-8836-829870c1351b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@387c8d02 2021-08-11T17:16:04.654Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=216d65cb-29e1-4120-8836-829870c1351b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:16:04.654Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=216d65cb-29e1-4120-8836-829870c1351b PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:16:07.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73f1e984-ecbc-41f3-8b7e-7cc69d15d001 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:16:07.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73f1e984-ecbc-41f3-8b7e-7cc69d15d001 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4374286c 2021-08-11T17:16:07.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73f1e984-ecbc-41f3-8b7e-7cc69d15d001 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:16:07.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73f1e984-ecbc-41f3-8b7e-7cc69d15d001 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:16:10.603Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e7502900-156b-4d11-9e56-7ed83b0e8932 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:16:10.603Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e7502900-156b-4d11-9e56-7ed83b0e8932 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@328ac15 2021-08-11T17:16:10.603Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e7502900-156b-4d11-9e56-7ed83b0e8932 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:16:10.603Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e7502900-156b-4d11-9e56-7ed83b0e8932 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:16:13.596Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9fd6f87-c398-4aab-b1b1-5171d28af2e0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:16:13.596Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9fd6f87-c398-4aab-b1b1-5171d28af2e0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@12f7d4f9 2021-08-11T17:16:13.596Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9fd6f87-c398-4aab-b1b1-5171d28af2e0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:16:13.596Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a9fd6f87-c398-4aab-b1b1-5171d28af2e0 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:16:16.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e190be13-88c8-45d1-a890-5577c207f8d5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:16:16.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e190be13-88c8-45d1-a890-5577c207f8d5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4e3f4f38 2021-08-11T17:16:16.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e190be13-88c8-45d1-a890-5577c207f8d5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:16:16.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e190be13-88c8-45d1-a890-5577c207f8d5 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:16:18.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:16:18.310Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:16:19.705Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=092b389f-bf53-4384-87f3-451823eb1809 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:16:19.705Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=092b389f-bf53-4384-87f3-451823eb1809 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2cea39db 2021-08-11T17:16:19.705Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=092b389f-bf53-4384-87f3-451823eb1809 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:16:19.705Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=092b389f-bf53-4384-87f3-451823eb1809 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:16:22.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=88aa1a44-b6af-4bda-9d1d-a99cb098a94b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:16:22.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=88aa1a44-b6af-4bda-9d1d-a99cb098a94b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6a646057 2021-08-11T17:16:22.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=88aa1a44-b6af-4bda-9d1d-a99cb098a94b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:16:22.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=88aa1a44-b6af-4bda-9d1d-a99cb098a94b PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:16:25.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9b113096-96c0-4bff-b902-f07820f57c72 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:16:25.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9b113096-96c0-4bff-b902-f07820f57c72 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@461821fe 2021-08-11T17:16:25.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9b113096-96c0-4bff-b902-f07820f57c72 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:16:25.680Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9b113096-96c0-4bff-b902-f07820f57c72 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:16:28.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5df54bb1-ca99-4b0e-bb65-b4c39c848054 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:16:28.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5df54bb1-ca99-4b0e-bb65-b4c39c848054 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@72eae08a 2021-08-11T17:16:28.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5df54bb1-ca99-4b0e-bb65-b4c39c848054 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:16:28.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5df54bb1-ca99-4b0e-bb65-b4c39c848054 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:16:31.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=184c122c-f333-47e1-beed-b985e839abd7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:16:31.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=184c122c-f333-47e1-beed-b985e839abd7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@f333bed 2021-08-11T17:16:31.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=184c122c-f333-47e1-beed-b985e839abd7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:16:31.674Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=184c122c-f333-47e1-beed-b985e839abd7 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:16:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:16:33.316Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:16:34.598Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0dc3a148-a848-4bc9-b382-61a068065fc2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:16:34.598Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0dc3a148-a848-4bc9-b382-61a068065fc2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@461c8484 2021-08-11T17:16:34.598Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0dc3a148-a848-4bc9-b382-61a068065fc2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:16:34.598Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0dc3a148-a848-4bc9-b382-61a068065fc2 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:16:37.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a65c1637-f971-401a-ba72-ee2bedf975ca PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:16:37.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a65c1637-f971-401a-ba72-ee2bedf975ca PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@418c4a68 2021-08-11T17:16:37.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a65c1637-f971-401a-ba72-ee2bedf975ca PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:16:37.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a65c1637-f971-401a-ba72-ee2bedf975ca PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:16:40.604Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b10e30f5-8f93-4dfb-b41a-da8bd4be9ec1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:16:40.604Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b10e30f5-8f93-4dfb-b41a-da8bd4be9ec1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@64c70c63 2021-08-11T17:16:40.604Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b10e30f5-8f93-4dfb-b41a-da8bd4be9ec1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:16:40.604Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b10e30f5-8f93-4dfb-b41a-da8bd4be9ec1 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:16:43.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d9c0c43-ce63-4cb8-a847-454e9fb4fcfc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:16:43.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d9c0c43-ce63-4cb8-a847-454e9fb4fcfc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@72bdbf45 2021-08-11T17:16:43.598Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d9c0c43-ce63-4cb8-a847-454e9fb4fcfc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:16:43.598Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6d9c0c43-ce63-4cb8-a847-454e9fb4fcfc PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:16:46.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a24352e-400e-4de7-adcd-79b1e4fecb35 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:16:46.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a24352e-400e-4de7-adcd-79b1e4fecb35 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@50273d32 2021-08-11T17:16:46.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a24352e-400e-4de7-adcd-79b1e4fecb35 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:16:46.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8a24352e-400e-4de7-adcd-79b1e4fecb35 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:16:48.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:16:48.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:16:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T17:16:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T17:16:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T17:16:50.150Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc069cd3-e7f7-4e68-9a4a-8918472db6eb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:16:50.150Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc069cd3-e7f7-4e68-9a4a-8918472db6eb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@735be254 2021-08-11T17:16:50.150Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc069cd3-e7f7-4e68-9a4a-8918472db6eb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:16:50.150Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dc069cd3-e7f7-4e68-9a4a-8918472db6eb PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:16:53.354Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7e52529-6cbc-492b-a3ae-ac33a93a0834 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:16:53.354Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7e52529-6cbc-492b-a3ae-ac33a93a0834 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2fa34c48 2021-08-11T17:16:53.354Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7e52529-6cbc-492b-a3ae-ac33a93a0834 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:16:53.354Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7e52529-6cbc-492b-a3ae-ac33a93a0834 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:16:56.545Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1774605-e090-4a93-b589-10ca88b30b47 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:16:56.545Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1774605-e090-4a93-b589-10ca88b30b47 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@30fb896e 2021-08-11T17:16:56.545Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1774605-e090-4a93-b589-10ca88b30b47 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:16:56.545Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1774605-e090-4a93-b589-10ca88b30b47 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:16:59.356Z [cluster1-nio-worker-1] WARN c.d.driver.core.HostConnectionPool Error creating connection to cassandra.onap/10.242.184.131:9042 com.datastax.driver.core.exceptions.TransportException: [cassandra.onap/10.242.184.131:9042] Connection has been closed at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1427) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1408) at com.datastax.driver.core.Connection.defunct(Connection.java:585) at com.datastax.driver.core.Connection$Dispatcher.exceptionCaught(Connection.java:1330) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273) at io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.handleReadException(AbstractEpollStreamChannel.java:728) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:818) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-08-11T17:16:59.356Z [cluster1-nio-worker-1] WARN com.datastax.driver.core.Session Error creating pool to cassandra.onap/10.242.184.131:9042 com.datastax.driver.core.exceptions.ConnectionException: [cassandra.onap/10.242.184.131:9042] Pool was closed during initialization at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:173) at com.datastax.driver.core.HostConnectionPool$2.onSuccess(HostConnectionPool.java:158) at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1080) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:746) at com.google.common.util.concurrent.CollectionFuture.handleAllCompleted(CollectionFuture.java:69) at com.google.common.util.concurrent.AggregateFuture.processCompleted(AggregateFuture.java:282) at com.google.common.util.concurrent.AggregateFuture.decrementCountAndMaybeComplete(AggregateFuture.java:264) at com.google.common.util.concurrent.AggregateFuture.access$200(AggregateFuture.java:42) at com.google.common.util.concurrent.AggregateFuture$1.run(AggregateFuture.java:147) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setFuture(AbstractFuture.java:814) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:203) at com.google.common.util.concurrent.AbstractCatchingFuture$AsyncCatchingFuture.setResult(AbstractCatchingFuture.java:179) at com.google.common.util.concurrent.AbstractCatchingFuture.run(AbstractCatchingFuture.java:133) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:100) at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1213) at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:983) at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:771) at com.datastax.driver.core.Connection$Future.onException(Connection.java:1518) at com.datastax.driver.core.Connection$Dispatcher.errorOutAllHandler(Connection.java:1340) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1427) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1408) at com.datastax.driver.core.Connection.defunct(Connection.java:585) at com.datastax.driver.core.Connection$Dispatcher.exceptionCaught(Connection.java:1330) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273) at io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.handleReadException(AbstractEpollStreamChannel.java:728) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:818) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-08-11T17:16:59.357Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad0b977a-fefc-4c74-9386-2ea5b183ce79 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:16:59.357Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad0b977a-fefc-4c74-9386-2ea5b183ce79 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@595cfcee 2021-08-11T17:16:59.357Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad0b977a-fefc-4c74-9386-2ea5b183ce79 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:16:59.357Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ad0b977a-fefc-4c74-9386-2ea5b183ce79 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:16:59.771Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=ad0b977a-fefc-4c74-9386-2ea5b183ce79 ErrorCategory=INFO ServiceName=BE Component CASSANDRA state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component CASSANDRA state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T17:16:59.772Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=ad0b977a-fefc-4c74-9386-2ea5b183ce79 ErrorCategory=ERROR ServiceName=BE Component CASSANDRA state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-08-11T17:17:01.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cca3f3d7-bb51-40b0-962c-e6ec499a71c4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:17:01.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cca3f3d7-bb51-40b0-962c-e6ec499a71c4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7539ecd3 2021-08-11T17:17:01.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cca3f3d7-bb51-40b0-962c-e6ec499a71c4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:17:01.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cca3f3d7-bb51-40b0-962c-e6ec499a71c4 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:17:02.250Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:02.277Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:02.330Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:02.433Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:02.635Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:03.037Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:03.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:17:03.245Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-08-11T17:17:03.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:17:03.310Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:17:03.358Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:03.410Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:03.440Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:03.463Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:03.515Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:03.617Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:03.820Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:04.022Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:04.225Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:04.243Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:04.254Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:17:04.427Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:04.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d742757d-faee-4b5f-a07f-39a12d4557eb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:17:04.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d742757d-faee-4b5f-a07f-39a12d4557eb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@404d3f30 2021-08-11T17:17:04.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d742757d-faee-4b5f-a07f-39a12d4557eb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:17:04.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d742757d-faee-4b5f-a07f-39a12d4557eb PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:17:04.593Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=d742757d-faee-4b5f-a07f-39a12d4557eb ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T17:17:04.593Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=d742757d-faee-4b5f-a07f-39a12d4557eb ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-08-11T17:17:04.630Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:05.032Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:05.835Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T17:16:57.848795Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:05.845Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:06.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:17:07.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=84f490db-80c7-4900-be0b-f20cfa0e7f53 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:17:07.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=84f490db-80c7-4900-be0b-f20cfa0e7f53 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2c917eae 2021-08-11T17:17:07.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=84f490db-80c7-4900-be0b-f20cfa0e7f53 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:17:07.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=84f490db-80c7-4900-be0b-f20cfa0e7f53 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:17:09.050Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:09.077Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:09.115Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:09.142Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:09.169Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:09.196Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:09.224Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:09.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:17:09.251Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:09.278Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:09.305Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:09.357Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:09.460Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:09.662Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:10.065Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:10.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d0eed08f-162d-4ff3-a06a-a6045ce1889d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:17:10.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d0eed08f-162d-4ff3-a06a-a6045ce1889d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@12144c5e 2021-08-11T17:17:10.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d0eed08f-162d-4ff3-a06a-a6045ce1889d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:17:10.589Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d0eed08f-162d-4ff3-a06a-a6045ce1889d PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:17:10.837Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:10.865Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:10.866Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:10.892Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:10.919Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:10.971Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:11.024Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:11.127Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:11.330Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:11.733Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:12.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:17:12.469Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:12.536Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:13.338Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:13.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b94cb2a-47e5-4d55-8884-f06a09f3df88 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:17:13.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b94cb2a-47e5-4d55-8884-f06a09f3df88 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@287165bb 2021-08-11T17:17:13.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b94cb2a-47e5-4d55-8884-f06a09f3df88 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:17:13.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4b94cb2a-47e5-4d55-8884-f06a09f3df88 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:17:14.072Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:14.145Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T17:16:57.848795Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:15.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:17:15.675Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:16.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0f8900d-f468-4a17-820a-e915db6eccb9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:17:16.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0f8900d-f468-4a17-820a-e915db6eccb9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1ec6b3a 2021-08-11T17:17:16.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0f8900d-f468-4a17-820a-e915db6eccb9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:17:16.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0f8900d-f468-4a17-820a-e915db6eccb9 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:17:17.278Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:18.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:17:18.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:17:18.316Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:17:18.885Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:18.912Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:18.939Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:18.967Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:18.994Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:19.046Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:19.147Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:19.148Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:19.199Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:19.252Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:19.351Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:19.354Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:19.456Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:19.554Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:19.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db37e518-0eb4-4fa9-8e72-32ecbbe17530 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:17:19.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db37e518-0eb4-4fa9-8e72-32ecbbe17530 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@262f01a7 2021-08-11T17:17:19.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db37e518-0eb4-4fa9-8e72-32ecbbe17530 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:17:19.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=db37e518-0eb4-4fa9-8e72-32ecbbe17530 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:17:19.658Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:19.861Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:19.956Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:20.063Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:20.267Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:20.670Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:20.759Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:21.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:17:21.472Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:22.362Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:22.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=373e53e3-2453-4272-85ce-24e06d055a19 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:17:22.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=373e53e3-2453-4272-85ce-24e06d055a19 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@16e9df80 2021-08-11T17:17:22.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=373e53e3-2453-4272-85ce-24e06d055a19 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:17:22.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=373e53e3-2453-4272-85ce-24e06d055a19 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:17:23.075Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T17:16:57.848795Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:23.965Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:24.245Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:17:25.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24917356-99b7-45bb-94ef-3d5a3204af52 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:17:25.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24917356-99b7-45bb-94ef-3d5a3204af52 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@35163c5b 2021-08-11T17:17:25.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24917356-99b7-45bb-94ef-3d5a3204af52 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:17:25.682Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=24917356-99b7-45bb-94ef-3d5a3204af52 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:17:27.170Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:27.198Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:27.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:17:27.248Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:27.275Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:27.348Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:27.375Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:27.448Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:27.476Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:27.550Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:27.649Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:27.702Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:27.847Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:27.950Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:28.052Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:28.078Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:28.147Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:28.154Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:28.251Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:28.257Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:28.348Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:28.448Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:28.462Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:28.550Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:28.653Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:28.855Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:28.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=132ce7cc-7b3a-4395-97ba-9c4a9bb25b7a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:17:28.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=132ce7cc-7b3a-4395-97ba-9c4a9bb25b7a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@57c12478 2021-08-11T17:17:28.860Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=132ce7cc-7b3a-4395-97ba-9c4a9bb25b7a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:17:28.861Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=132ce7cc-7b3a-4395-97ba-9c4a9bb25b7a PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:17:28.863Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:29.161Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:29.266Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:29.564Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:30.068Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:30.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:17:30.367Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:31.170Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T17:16:57.848795Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:31.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7b4f405-697a-4ee5-bdd5-ca769409ea81 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:17:31.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7b4f405-697a-4ee5-bdd5-ca769409ea81 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@93d251e 2021-08-11T17:17:31.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7b4f405-697a-4ee5-bdd5-ca769409ea81 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:17:31.579Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f7b4f405-697a-4ee5-bdd5-ca769409ea81 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:17:31.671Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:33.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:17:33.274Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:17:33.311Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:17:34.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2a5ad33-f285-415f-811c-f92e98fa0136 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:17:34.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2a5ad33-f285-415f-811c-f92e98fa0136 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5170ee66 2021-08-11T17:17:34.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2a5ad33-f285-415f-811c-f92e98fa0136 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:17:34.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f2a5ad33-f285-415f-811c-f92e98fa0136 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:17:34.879Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:34.906Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:34.933Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:34.960Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:35.013Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:35.115Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:35.218Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:35.321Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:35.523Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:35.726Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:35.928Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:36.131Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:36.172Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:36.225Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:36.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:17:36.327Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:36.429Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:36.531Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:36.534Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:36.734Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:37.136Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:37.337Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:37.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b54c47bd-de94-4f12-8324-dee511218a26 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:17:37.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b54c47bd-de94-4f12-8324-dee511218a26 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5580f4f7 2021-08-11T17:17:37.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b54c47bd-de94-4f12-8324-dee511218a26 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:17:37.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b54c47bd-de94-4f12-8324-dee511218a26 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:17:37.939Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:38.947Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:39.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:17:39.548Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T17:16:57.848795Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:40.853Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ba61dbc-0e9c-4493-aa7e-d54177c2780a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:17:40.853Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ba61dbc-0e9c-4493-aa7e-d54177c2780a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@bbcf8db 2021-08-11T17:17:40.853Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ba61dbc-0e9c-4493-aa7e-d54177c2780a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:17:40.853Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0ba61dbc-0e9c-4493-aa7e-d54177c2780a PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:17:42.153Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:42.180Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:42.233Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:42.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:17:42.285Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:42.337Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:42.440Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:42.643Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:42.846Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:43.249Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:43.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dcd66564-a2a9-4861-b922-a0da4795a678 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:17:43.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dcd66564-a2a9-4861-b922-a0da4795a678 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6b492967 2021-08-11T17:17:43.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dcd66564-a2a9-4861-b922-a0da4795a678 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:17:43.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dcd66564-a2a9-4861-b922-a0da4795a678 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:17:43.651Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:44.454Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:44.555Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:44.583Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:44.610Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:44.637Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:44.689Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:44.741Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:44.795Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:17:45.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:17:46.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e8c19ff-acd7-40eb-854f-50e17304851b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:17:46.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e8c19ff-acd7-40eb-854f-50e17304851b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3d9cf9b6 2021-08-11T17:17:46.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e8c19ff-acd7-40eb-854f-50e17304851b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:17:46.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e8c19ff-acd7-40eb-854f-50e17304851b PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:17:47.152Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=4e8c19ff-acd7-40eb-854f-50e17304851b ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T17:17:47.152Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=4e8c19ff-acd7-40eb-854f-50e17304851b ErrorCategory=ERROR ServiceName=BE Component ON_BOARDING state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-08-11T17:17:47.265Z [JanusGraph-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck ASDC Backend connection recovery to JanusGraph DB. 2021-08-11T17:17:48.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:17:48.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:17:49.290Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T17:17:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T17:17:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T17:17:49.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0fe52c9-24d9-4081-8da8-1b3ec0db6c0f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:17:49.582Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0fe52c9-24d9-4081-8da8-1b3ec0db6c0f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6fe985be 2021-08-11T17:17:49.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0fe52c9-24d9-4081-8da8-1b3ec0db6c0f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:17:49.583Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0fe52c9-24d9-4081-8da8-1b3ec0db6c0f PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:17:49.865Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=c0fe52c9-24d9-4081-8da8-1b3ec0db6c0f ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T17:17:49.865Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=c0fe52c9-24d9-4081-8da8-1b3ec0db6c0f ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-08-11T17:17:52.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=39ddff90-7c08-4f7d-a3a9-ce9b134e2d8e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:17:52.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=39ddff90-7c08-4f7d-a3a9-ce9b134e2d8e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3ffc2b8 2021-08-11T17:17:52.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=39ddff90-7c08-4f7d-a3a9-ce9b134e2d8e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:17:52.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=39ddff90-7c08-4f7d-a3a9-ce9b134e2d8e PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:17:55.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b600a0b-175e-4527-a000-c6c6cf83af51 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:17:55.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b600a0b-175e-4527-a000-c6c6cf83af51 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@21dfc1b1 2021-08-11T17:17:55.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b600a0b-175e-4527-a000-c6c6cf83af51 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:17:55.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1b600a0b-175e-4527-a000-c6c6cf83af51 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:18:01.559Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0af74ad-87dc-4302-a431-50a3491fc7ba PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:18:01.560Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0af74ad-87dc-4302-a431-50a3491fc7ba PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7bd4cd75 2021-08-11T17:18:01.560Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0af74ad-87dc-4302-a431-50a3491fc7ba PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:18:01.560Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0af74ad-87dc-4302-a431-50a3491fc7ba PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:18:03.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:18:03.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:18:03.872Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=c0af74ad-87dc-4302-a431-50a3491fc7ba ErrorCategory=INFO ServiceName=BE Component CASSANDRA state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component CASSANDRA state changed from DOWN to UP ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T17:18:03.873Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=c0af74ad-87dc-4302-a431-50a3491fc7ba ErrorCategory=INFO ServiceName=BE Component CASSANDRA state changed from DOWN to UP TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T17:18:04.061Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6cd4e46f-6dfe-4c1b-8b8f-58f27987b543 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:18:04.061Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6cd4e46f-6dfe-4c1b-8b8f-58f27987b543 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@58b363c4 2021-08-11T17:18:04.061Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6cd4e46f-6dfe-4c1b-8b8f-58f27987b543 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:18:04.061Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6cd4e46f-6dfe-4c1b-8b8f-58f27987b543 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:18:04.276Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:18:04.878Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df586605-3463-47e4-a785-36ef144ee43b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:18:04.878Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df586605-3463-47e4-a785-36ef144ee43b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@25248068 2021-08-11T17:18:04.878Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df586605-3463-47e4-a785-36ef144ee43b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:18:04.878Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=df586605-3463-47e4-a785-36ef144ee43b PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:18:07.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4781bd9a-6d4a-4077-8f2a-87a4b3c878b0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:18:07.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4781bd9a-6d4a-4077-8f2a-87a4b3c878b0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7aa91f34 2021-08-11T17:18:07.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4781bd9a-6d4a-4077-8f2a-87a4b3c878b0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:18:07.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4781bd9a-6d4a-4077-8f2a-87a4b3c878b0 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:18:10.851Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9bc5a0a-d007-4a78-ae37-f3cea5fc6e62 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:18:10.851Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9bc5a0a-d007-4a78-ae37-f3cea5fc6e62 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@274bca71 2021-08-11T17:18:10.851Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9bc5a0a-d007-4a78-ae37-f3cea5fc6e62 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:18:10.851Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e9bc5a0a-d007-4a78-ae37-f3cea5fc6e62 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:18:13.652Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a1fab50-bf2d-4a86-8bd8-fb98fb8653e3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:18:13.652Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a1fab50-bf2d-4a86-8bd8-fb98fb8653e3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6425201f 2021-08-11T17:18:13.652Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a1fab50-bf2d-4a86-8bd8-fb98fb8653e3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:18:13.652Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6a1fab50-bf2d-4a86-8bd8-fb98fb8653e3 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:18:16.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3a3f63db-0203-4344-90f1-062592b2c9bf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:18:16.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3a3f63db-0203-4344-90f1-062592b2c9bf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6503db33 2021-08-11T17:18:16.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3a3f63db-0203-4344-90f1-062592b2c9bf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:18:16.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3a3f63db-0203-4344-90f1-062592b2c9bf PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:18:18.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:18:18.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:18:19.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1faeaa6e-1d2d-47c6-a406-91f63028812f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:18:19.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1faeaa6e-1d2d-47c6-a406-91f63028812f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@57612818 2021-08-11T17:18:19.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1faeaa6e-1d2d-47c6-a406-91f63028812f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:18:19.671Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1faeaa6e-1d2d-47c6-a406-91f63028812f PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:18:22.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=16cf9059-783b-42cf-94e8-1d171c2abbfb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:18:22.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=16cf9059-783b-42cf-94e8-1d171c2abbfb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2e0068c7 2021-08-11T17:18:22.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=16cf9059-783b-42cf-94e8-1d171c2abbfb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:18:22.684Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=16cf9059-783b-42cf-94e8-1d171c2abbfb PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:18:25.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d47b1b14-4cc6-49cb-a19a-f030f5476fb4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:18:25.763Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d47b1b14-4cc6-49cb-a19a-f030f5476fb4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@63e880c0 2021-08-11T17:18:25.764Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d47b1b14-4cc6-49cb-a19a-f030f5476fb4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:18:25.764Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d47b1b14-4cc6-49cb-a19a-f030f5476fb4 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:18:28.959Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e7014a3-b756-45d3-9faf-01afc14d6a2b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:18:28.959Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e7014a3-b756-45d3-9faf-01afc14d6a2b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@18c915cd 2021-08-11T17:18:28.959Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e7014a3-b756-45d3-9faf-01afc14d6a2b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:18:28.959Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e7014a3-b756-45d3-9faf-01afc14d6a2b PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:18:31.802Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=051d9d7f-a23a-4e03-bea1-d7ab37df4d45 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:18:31.802Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=051d9d7f-a23a-4e03-bea1-d7ab37df4d45 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4636cf24 2021-08-11T17:18:31.802Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=051d9d7f-a23a-4e03-bea1-d7ab37df4d45 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:18:31.802Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=051d9d7f-a23a-4e03-bea1-d7ab37df4d45 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:18:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:18:33.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:18:34.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f9eb029-91bb-414c-9b12-41cffabb6e48 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:18:34.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f9eb029-91bb-414c-9b12-41cffabb6e48 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6bddd175 2021-08-11T17:18:34.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f9eb029-91bb-414c-9b12-41cffabb6e48 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:18:34.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f9eb029-91bb-414c-9b12-41cffabb6e48 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:18:37.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85d66db1-a57d-4e7e-9053-9bce822de59b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:18:37.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85d66db1-a57d-4e7e-9053-9bce822de59b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4c14a24e 2021-08-11T17:18:37.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85d66db1-a57d-4e7e-9053-9bce822de59b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:18:37.662Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85d66db1-a57d-4e7e-9053-9bce822de59b PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:18:40.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=345e91b9-4261-4bcb-adf6-b6bfee21cbfe PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:18:40.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=345e91b9-4261-4bcb-adf6-b6bfee21cbfe PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1ca3a17c 2021-08-11T17:18:40.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=345e91b9-4261-4bcb-adf6-b6bfee21cbfe PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:18:40.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=345e91b9-4261-4bcb-adf6-b6bfee21cbfe PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:18:43.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=22ab0fba-89f6-45f9-98b8-c9df8708fc2f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:18:43.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=22ab0fba-89f6-45f9-98b8-c9df8708fc2f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6c1c0223 2021-08-11T17:18:43.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=22ab0fba-89f6-45f9-98b8-c9df8708fc2f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:18:43.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=22ab0fba-89f6-45f9-98b8-c9df8708fc2f PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:18:46.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b45a464c-68a1-41a8-88f6-cfe29629f64b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:18:46.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b45a464c-68a1-41a8-88f6-cfe29629f64b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@694f30ac 2021-08-11T17:18:46.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b45a464c-68a1-41a8-88f6-cfe29629f64b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:18:46.624Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b45a464c-68a1-41a8-88f6-cfe29629f64b PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:18:48.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:18:48.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:18:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T17:18:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T17:18:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T17:18:49.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d47ed781-c72f-41b9-a4cd-cc6847f73856 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:18:49.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d47ed781-c72f-41b9-a4cd-cc6847f73856 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@20f919 2021-08-11T17:18:49.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d47ed781-c72f-41b9-a4cd-cc6847f73856 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:18:49.646Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d47ed781-c72f-41b9-a4cd-cc6847f73856 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:18:52.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11bd3c4c-629d-4a8a-b958-a3bb7e378886 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:18:52.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11bd3c4c-629d-4a8a-b958-a3bb7e378886 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1890020 2021-08-11T17:18:52.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11bd3c4c-629d-4a8a-b958-a3bb7e378886 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:18:52.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=11bd3c4c-629d-4a8a-b958-a3bb7e378886 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:18:55.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d15c7139-3841-44bd-87bb-e26008f66daa PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:18:55.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d15c7139-3841-44bd-87bb-e26008f66daa PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@d880728 2021-08-11T17:18:55.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d15c7139-3841-44bd-87bb-e26008f66daa PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:18:55.767Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d15c7139-3841-44bd-87bb-e26008f66daa PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:18:58.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=995f7e9e-b269-4a4b-aba2-82391b0143e7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:18:58.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=995f7e9e-b269-4a4b-aba2-82391b0143e7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1dd28c65 2021-08-11T17:18:58.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=995f7e9e-b269-4a4b-aba2-82391b0143e7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:18:58.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=995f7e9e-b269-4a4b-aba2-82391b0143e7 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:19:01.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=48eb2d7b-b399-473a-94b8-70224c336e00 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:19:01.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=48eb2d7b-b399-473a-94b8-70224c336e00 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1abd56f7 2021-08-11T17:19:01.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=48eb2d7b-b399-473a-94b8-70224c336e00 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:19:01.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=48eb2d7b-b399-473a-94b8-70224c336e00 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:19:03.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:19:03.317Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:19:04.247Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:19:04.688Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66cfe640-0edc-466f-8383-82c16cc84e5f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:19:04.688Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66cfe640-0edc-466f-8383-82c16cc84e5f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@53fccd80 2021-08-11T17:19:04.688Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66cfe640-0edc-466f-8383-82c16cc84e5f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:19:04.688Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=66cfe640-0edc-466f-8383-82c16cc84e5f PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:19:07.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dcd2a329-7cb2-4b64-b438-c3b5709aa993 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:19:07.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dcd2a329-7cb2-4b64-b438-c3b5709aa993 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1e803190 2021-08-11T17:19:07.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dcd2a329-7cb2-4b64-b438-c3b5709aa993 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:19:07.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dcd2a329-7cb2-4b64-b438-c3b5709aa993 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:19:10.618Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9616e34b-c6c0-40cc-a068-cb7bb7b2dacb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:19:10.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9616e34b-c6c0-40cc-a068-cb7bb7b2dacb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@17b83781 2021-08-11T17:19:10.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9616e34b-c6c0-40cc-a068-cb7bb7b2dacb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:19:10.619Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9616e34b-c6c0-40cc-a068-cb7bb7b2dacb PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:19:13.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b795b9ba-6a56-4ebb-ac9a-b95cdd880f43 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:19:13.774Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b795b9ba-6a56-4ebb-ac9a-b95cdd880f43 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@67c2d16f 2021-08-11T17:19:13.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b795b9ba-6a56-4ebb-ac9a-b95cdd880f43 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:19:13.775Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b795b9ba-6a56-4ebb-ac9a-b95cdd880f43 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:19:16.845Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7071d131-bf36-4f45-85e3-78046e6d4e4d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:19:16.846Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7071d131-bf36-4f45-85e3-78046e6d4e4d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@52e780da 2021-08-11T17:19:16.846Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7071d131-bf36-4f45-85e3-78046e6d4e4d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:19:16.846Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7071d131-bf36-4f45-85e3-78046e6d4e4d PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:19:18.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:19:18.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:19:19.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf606f65-65b6-4a74-9273-1b640ccccbe6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:19:19.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf606f65-65b6-4a74-9273-1b640ccccbe6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4d7427a7 2021-08-11T17:19:19.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf606f65-65b6-4a74-9273-1b640ccccbe6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:19:19.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bf606f65-65b6-4a74-9273-1b640ccccbe6 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:19:22.611Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac2ee9a1-3da9-4f12-aaf7-5e6fcb086e24 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:19:22.611Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac2ee9a1-3da9-4f12-aaf7-5e6fcb086e24 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1ba615b7 2021-08-11T17:19:22.612Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac2ee9a1-3da9-4f12-aaf7-5e6fcb086e24 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:19:22.612Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ac2ee9a1-3da9-4f12-aaf7-5e6fcb086e24 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:19:25.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=93ce8a8c-6d1d-4f54-bef9-662b7d863415 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:19:25.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=93ce8a8c-6d1d-4f54-bef9-662b7d863415 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6a9ee605 2021-08-11T17:19:25.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=93ce8a8c-6d1d-4f54-bef9-662b7d863415 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:19:25.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=93ce8a8c-6d1d-4f54-bef9-662b7d863415 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:19:28.949Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd15dbd8-bbe9-45b9-99a4-71b77791aeb2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:19:28.949Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd15dbd8-bbe9-45b9-99a4-71b77791aeb2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4914adc 2021-08-11T17:19:28.949Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd15dbd8-bbe9-45b9-99a4-71b77791aeb2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:19:28.950Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dd15dbd8-bbe9-45b9-99a4-71b77791aeb2 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:19:29.343Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01w3a77h2ngqj1cb72ka1etnql255 2021-08-11T17:19:29.343Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01qpxjgnxyqk6n1uprp232ty7ak258 2021-08-11T17:19:29.343Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01owo7e6x1llckenv0omx91tzu265 2021-08-11T17:19:29.343Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0m76qjzi72vi5uin7nlexkpf7264 2021-08-11T17:19:29.343Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node010sswky2r13hf18r686rokj7ez247 2021-08-11T17:19:29.343Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node02141h581j2931isnxxuw6jw9z246 2021-08-11T17:19:29.343Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node03t177bp1ok2f1smtt3vp90z1k253 2021-08-11T17:19:29.343Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01w094h13a32o1g1vga2oi8gf9254 2021-08-11T17:19:29.343Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0j2y7ni34yq1z1vpkevzt3rvqo267 2021-08-11T17:19:29.343Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0kups8ooowjmf1rq714tc7gsxw248 2021-08-11T17:19:29.344Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01l9409fgdn21v18y7ink2r6a2l261 2021-08-11T17:19:29.345Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0ep7ihreu0eqi1pet2xwj2b8p0252 2021-08-11T17:19:29.345Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0w7kq6mqt7bn58rumwyg1ubt0251 2021-08-11T17:19:29.345Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01dmz29anxuf0o1h3byzgwh3eku249 2021-08-11T17:19:29.345Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01gl6436gyloartawm77zppij8250 2021-08-11T17:19:29.345Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node09ku572jqm8iu10y2f5ztyayg3266 2021-08-11T17:19:29.345Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0nzvbs5wnbhv61iaieps0fb7k0256 2021-08-11T17:19:29.345Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01rer1yk35hh64ps3gwyrlj3v257 2021-08-11T17:19:29.345Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0144pcdx48eemu5puakouzxbpk259 2021-08-11T17:19:29.345Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0znkt5u0l59oc0itw5gymci263 2021-08-11T17:19:29.345Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node0yf7s46rc8kfmgc7ltg4ywkee260 2021-08-11T17:19:29.345Z [Scheduler-2073707154-1] INFO o.o.p.c.o.l.PortalTimeoutHandler Session getting destroyed - id: node01e3xbkqumdia417w8uugiftobn262 2021-08-11T17:19:31.698Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e0a945f-eaa3-4e4c-8ba0-014a1950e223 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:19:31.699Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e0a945f-eaa3-4e4c-8ba0-014a1950e223 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@68341cf6 2021-08-11T17:19:31.699Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e0a945f-eaa3-4e4c-8ba0-014a1950e223 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:19:31.699Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2e0a945f-eaa3-4e4c-8ba0-014a1950e223 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:19:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:19:33.311Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:19:34.610Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=92c89ced-9228-4b2b-b226-bce3236a95ba PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:19:34.611Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=92c89ced-9228-4b2b-b226-bce3236a95ba PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4394a7ea 2021-08-11T17:19:34.611Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=92c89ced-9228-4b2b-b226-bce3236a95ba PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:19:34.611Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=92c89ced-9228-4b2b-b226-bce3236a95ba PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:19:37.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e6a1356-16ec-4f1c-95a4-9d7c0aab5b49 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:19:37.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e6a1356-16ec-4f1c-95a4-9d7c0aab5b49 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5e27cf53 2021-08-11T17:19:37.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e6a1356-16ec-4f1c-95a4-9d7c0aab5b49 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:19:37.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0e6a1356-16ec-4f1c-95a4-9d7c0aab5b49 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:19:40.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01624a1f-a61e-4a3c-ad6f-66adc0100c27 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:19:40.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01624a1f-a61e-4a3c-ad6f-66adc0100c27 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@506ea9e1 2021-08-11T17:19:40.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01624a1f-a61e-4a3c-ad6f-66adc0100c27 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:19:40.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=01624a1f-a61e-4a3c-ad6f-66adc0100c27 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:19:43.705Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95c0b89b-ccee-4602-b5f3-3e41846503e2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:19:43.705Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95c0b89b-ccee-4602-b5f3-3e41846503e2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@18f5ccb6 2021-08-11T17:19:43.705Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95c0b89b-ccee-4602-b5f3-3e41846503e2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:19:43.705Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=95c0b89b-ccee-4602-b5f3-3e41846503e2 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:19:46.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a518ca7-b8ae-426c-9e97-3d79efe1fbea PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:19:46.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a518ca7-b8ae-426c-9e97-3d79efe1fbea PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7eb883b5 2021-08-11T17:19:46.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a518ca7-b8ae-426c-9e97-3d79efe1fbea PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:19:46.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9a518ca7-b8ae-426c-9e97-3d79efe1fbea PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:19:48.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:19:48.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:19:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T17:19:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T17:19:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T17:19:49.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18447d18-4068-4219-b51e-85ece2e0b2b3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:19:49.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18447d18-4068-4219-b51e-85ece2e0b2b3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1e858666 2021-08-11T17:19:49.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18447d18-4068-4219-b51e-85ece2e0b2b3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:19:49.868Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=18447d18-4068-4219-b51e-85ece2e0b2b3 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:19:53.149Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7f2fb6aa-883d-4ac3-8b43-6b776dbdab81 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:19:53.149Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7f2fb6aa-883d-4ac3-8b43-6b776dbdab81 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3b0105f6 2021-08-11T17:19:53.149Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7f2fb6aa-883d-4ac3-8b43-6b776dbdab81 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:19:53.149Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7f2fb6aa-883d-4ac3-8b43-6b776dbdab81 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:19:56.259Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5434aaff-5d14-498b-ab66-664d165c6e1f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:19:56.259Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5434aaff-5d14-498b-ab66-664d165c6e1f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@da7fe5c 2021-08-11T17:19:56.259Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5434aaff-5d14-498b-ab66-664d165c6e1f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:19:56.259Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5434aaff-5d14-498b-ab66-664d165c6e1f PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:19:58.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1542bc4-14bc-4cd6-8b80-f9581933f6d8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:19:58.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1542bc4-14bc-4cd6-8b80-f9581933f6d8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@37b86176 2021-08-11T17:19:58.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1542bc4-14bc-4cd6-8b80-f9581933f6d8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:19:58.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e1542bc4-14bc-4cd6-8b80-f9581933f6d8 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:20:01.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f98308b3-1f95-446c-a27b-8e6c9efb111d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:20:01.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f98308b3-1f95-446c-a27b-8e6c9efb111d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@53c6751 2021-08-11T17:20:01.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f98308b3-1f95-446c-a27b-8e6c9efb111d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:20:01.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f98308b3-1f95-446c-a27b-8e6c9efb111d PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:20:03.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:20:03.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:20:04.257Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:20:04.584Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70ec4ab1-216b-4414-8ac8-a1fb936bc5d3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:20:04.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70ec4ab1-216b-4414-8ac8-a1fb936bc5d3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@36a1fcb5 2021-08-11T17:20:04.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70ec4ab1-216b-4414-8ac8-a1fb936bc5d3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:20:04.585Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=70ec4ab1-216b-4414-8ac8-a1fb936bc5d3 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:20:07.617Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0563021-ab85-4286-9a9d-4333ff11060e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:20:07.617Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0563021-ab85-4286-9a9d-4333ff11060e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@446a123 2021-08-11T17:20:07.617Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0563021-ab85-4286-9a9d-4333ff11060e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:20:07.617Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0563021-ab85-4286-9a9d-4333ff11060e PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:20:10.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4355eafa-9e18-4999-b07e-497c90c44022 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:20:10.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4355eafa-9e18-4999-b07e-497c90c44022 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7001c30c 2021-08-11T17:20:10.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4355eafa-9e18-4999-b07e-497c90c44022 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:20:10.588Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4355eafa-9e18-4999-b07e-497c90c44022 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:20:13.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2aa2328a-2232-46df-aaa8-dfb539150a83 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:20:13.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2aa2328a-2232-46df-aaa8-dfb539150a83 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@16e134b2 2021-08-11T17:20:13.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2aa2328a-2232-46df-aaa8-dfb539150a83 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:20:13.622Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2aa2328a-2232-46df-aaa8-dfb539150a83 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:20:16.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14e09e78-0071-4a1c-b9bf-96754d6e6eaf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:20:16.608Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14e09e78-0071-4a1c-b9bf-96754d6e6eaf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2f1c268f 2021-08-11T17:20:16.608Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14e09e78-0071-4a1c-b9bf-96754d6e6eaf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:20:16.608Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14e09e78-0071-4a1c-b9bf-96754d6e6eaf PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:20:18.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:20:18.310Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:20:19.599Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5825159d-b914-4ac7-b821-4fec60a91798 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:20:19.599Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5825159d-b914-4ac7-b821-4fec60a91798 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1e62c5f6 2021-08-11T17:20:19.599Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5825159d-b914-4ac7-b821-4fec60a91798 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:20:19.599Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=5825159d-b914-4ac7-b821-4fec60a91798 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:20:22.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8ac37f9f-6418-494f-9191-11f8e6cc6509 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:20:22.663Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8ac37f9f-6418-494f-9191-11f8e6cc6509 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@60856c95 2021-08-11T17:20:22.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8ac37f9f-6418-494f-9191-11f8e6cc6509 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:20:22.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8ac37f9f-6418-494f-9191-11f8e6cc6509 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:20:25.747Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32110f43-6b45-4d8e-8206-d1747275b113 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:20:25.747Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32110f43-6b45-4d8e-8206-d1747275b113 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7fde8093 2021-08-11T17:20:25.747Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32110f43-6b45-4d8e-8206-d1747275b113 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:20:25.747Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=32110f43-6b45-4d8e-8206-d1747275b113 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:20:28.851Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c1cb732a-66a2-42f5-af1e-4603e7822ef3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:20:28.851Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c1cb732a-66a2-42f5-af1e-4603e7822ef3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@31b8f9b9 2021-08-11T17:20:28.851Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c1cb732a-66a2-42f5-af1e-4603e7822ef3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:20:28.852Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c1cb732a-66a2-42f5-af1e-4603e7822ef3 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:20:31.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=81369696-3d54-4561-8d5e-b47adf5defb8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:20:31.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=81369696-3d54-4561-8d5e-b47adf5defb8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@397dc70d 2021-08-11T17:20:31.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=81369696-3d54-4561-8d5e-b47adf5defb8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:20:31.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=81369696-3d54-4561-8d5e-b47adf5defb8 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:20:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:20:33.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:20:34.609Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0708fa41-41fa-4176-9cf9-2eb20bae1a29 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:20:34.609Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0708fa41-41fa-4176-9cf9-2eb20bae1a29 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@72d7f573 2021-08-11T17:20:34.609Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0708fa41-41fa-4176-9cf9-2eb20bae1a29 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:20:34.609Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0708fa41-41fa-4176-9cf9-2eb20bae1a29 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:20:37.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=00e28444-6eab-4c2e-9b01-56b941ea6d53 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:20:37.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=00e28444-6eab-4c2e-9b01-56b941ea6d53 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@798629a2 2021-08-11T17:20:37.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=00e28444-6eab-4c2e-9b01-56b941ea6d53 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:20:37.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=00e28444-6eab-4c2e-9b01-56b941ea6d53 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:20:40.587Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=27556668-88dc-470d-8bdc-9b7b76d3c7fb PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:20:40.587Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=27556668-88dc-470d-8bdc-9b7b76d3c7fb PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@f05621a 2021-08-11T17:20:40.587Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=27556668-88dc-470d-8bdc-9b7b76d3c7fb PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:20:40.587Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=27556668-88dc-470d-8bdc-9b7b76d3c7fb PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:20:43.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb89fba5-2f1e-4a50-a916-b66dfb378399 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:20:43.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb89fba5-2f1e-4a50-a916-b66dfb378399 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@58179958 2021-08-11T17:20:43.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb89fba5-2f1e-4a50-a916-b66dfb378399 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:20:43.645Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=cb89fba5-2f1e-4a50-a916-b66dfb378399 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:20:46.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d1252500-a096-4ee9-bbd1-e9b8368e2655 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:20:46.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d1252500-a096-4ee9-bbd1-e9b8368e2655 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@15269344 2021-08-11T17:20:46.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d1252500-a096-4ee9-bbd1-e9b8368e2655 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:20:46.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d1252500-a096-4ee9-bbd1-e9b8368e2655 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:20:48.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:20:48.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:20:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T17:20:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T17:20:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T17:20:50.358Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2fe61fcf-9fed-4ca5-873b-f7235303d8c8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:20:50.358Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2fe61fcf-9fed-4ca5-873b-f7235303d8c8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5da5737b 2021-08-11T17:20:50.358Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2fe61fcf-9fed-4ca5-873b-f7235303d8c8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:20:50.358Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2fe61fcf-9fed-4ca5-873b-f7235303d8c8 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:20:54.049Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0f43759-3f96-4358-9c75-a59881a2f5a0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:20:54.049Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0f43759-3f96-4358-9c75-a59881a2f5a0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7914cf67 2021-08-11T17:20:54.049Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0f43759-3f96-4358-9c75-a59881a2f5a0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:20:54.049Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e0f43759-3f96-4358-9c75-a59881a2f5a0 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:20:56.750Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14f47bb9-fe0d-4118-9539-ad1cc033743a PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:20:56.750Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14f47bb9-fe0d-4118-9539-ad1cc033743a PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1dad4444 2021-08-11T17:20:56.750Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14f47bb9-fe0d-4118-9539-ad1cc033743a PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:20:56.750Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14f47bb9-fe0d-4118-9539-ad1cc033743a PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:20:59.552Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=163fa91d-8c14-4a50-8cb3-18a3e4eee8e0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:20:59.553Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=163fa91d-8c14-4a50-8cb3-18a3e4eee8e0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3cb04806 2021-08-11T17:20:59.553Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=163fa91d-8c14-4a50-8cb3-18a3e4eee8e0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:20:59.553Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=163fa91d-8c14-4a50-8cb3-18a3e4eee8e0 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:21:02.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1eb32f4f-dfd8-4795-9252-b1765195d556 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:21:02.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1eb32f4f-dfd8-4795-9252-b1765195d556 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4a78ca40 2021-08-11T17:21:02.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1eb32f4f-dfd8-4795-9252-b1765195d556 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:21:02.073Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1eb32f4f-dfd8-4795-9252-b1765195d556 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:21:03.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:21:03.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:21:04.248Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:21:04.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=60fb4e9c-cad8-4ba6-8b6a-c870c90a8c6f PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:21:04.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=60fb4e9c-cad8-4ba6-8b6a-c870c90a8c6f PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@15c85b9f 2021-08-11T17:21:04.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=60fb4e9c-cad8-4ba6-8b6a-c870c90a8c6f PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:21:04.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=60fb4e9c-cad8-4ba6-8b6a-c870c90a8c6f PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:21:07.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e986bf05-acfb-4712-8ae1-14358bba3a85 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:21:07.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e986bf05-acfb-4712-8ae1-14358bba3a85 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6230a206 2021-08-11T17:21:07.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e986bf05-acfb-4712-8ae1-14358bba3a85 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:21:07.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e986bf05-acfb-4712-8ae1-14358bba3a85 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:21:10.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85abe24c-782a-4129-aa58-1cb838c24dae PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:21:10.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85abe24c-782a-4129-aa58-1cb838c24dae PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5cfe363e 2021-08-11T17:21:10.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85abe24c-782a-4129-aa58-1cb838c24dae PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:21:10.678Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=85abe24c-782a-4129-aa58-1cb838c24dae PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:21:13.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14ee3aa3-c54d-4c62-9ea6-c70772944945 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:21:13.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14ee3aa3-c54d-4c62-9ea6-c70772944945 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5058cd2b 2021-08-11T17:21:13.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14ee3aa3-c54d-4c62-9ea6-c70772944945 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:21:13.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=14ee3aa3-c54d-4c62-9ea6-c70772944945 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:21:16.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=163d16d5-6f6c-4682-a081-acca3cf79d65 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:21:16.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=163d16d5-6f6c-4682-a081-acca3cf79d65 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@79b701c 2021-08-11T17:21:16.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=163d16d5-6f6c-4682-a081-acca3cf79d65 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:21:16.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=163d16d5-6f6c-4682-a081-acca3cf79d65 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:21:18.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:21:18.316Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:21:19.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6de11543-0f12-4a6a-b39f-5ad97d9f70b7 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:21:19.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6de11543-0f12-4a6a-b39f-5ad97d9f70b7 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@60dcd27b 2021-08-11T17:21:19.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6de11543-0f12-4a6a-b39f-5ad97d9f70b7 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:21:19.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6de11543-0f12-4a6a-b39f-5ad97d9f70b7 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:21:22.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79015302-3c38-4444-8512-66a9271f7891 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:21:22.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79015302-3c38-4444-8512-66a9271f7891 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4dbe4798 2021-08-11T17:21:22.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79015302-3c38-4444-8512-66a9271f7891 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:21:22.675Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=79015302-3c38-4444-8512-66a9271f7891 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:21:25.749Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bbe95d0f-fa0d-4065-93b3-0534427ee7f0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:21:25.749Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bbe95d0f-fa0d-4065-93b3-0534427ee7f0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5b19fa91 2021-08-11T17:21:25.749Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bbe95d0f-fa0d-4065-93b3-0534427ee7f0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:21:25.749Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bbe95d0f-fa0d-4065-93b3-0534427ee7f0 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:21:28.951Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b49ac993-e859-4dff-bff4-68dd726f88a6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:21:28.952Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b49ac993-e859-4dff-bff4-68dd726f88a6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@75e7a7e 2021-08-11T17:21:28.952Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b49ac993-e859-4dff-bff4-68dd726f88a6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:21:28.952Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b49ac993-e859-4dff-bff4-68dd726f88a6 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:21:31.656Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f11708d9-0d23-4afe-a1a2-1256c3236cf6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:21:31.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f11708d9-0d23-4afe-a1a2-1256c3236cf6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1173fe4 2021-08-11T17:21:31.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f11708d9-0d23-4afe-a1a2-1256c3236cf6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:21:31.657Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f11708d9-0d23-4afe-a1a2-1256c3236cf6 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:21:33.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:21:33.316Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:21:34.650Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f20f9875-b0fc-4e71-91bb-662afe3979c5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:21:34.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f20f9875-b0fc-4e71-91bb-662afe3979c5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3eaa3ad9 2021-08-11T17:21:34.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f20f9875-b0fc-4e71-91bb-662afe3979c5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:21:34.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f20f9875-b0fc-4e71-91bb-662afe3979c5 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:21:37.754Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b4388089-ff26-43db-a740-8e56b060dcb6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:21:37.754Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b4388089-ff26-43db-a740-8e56b060dcb6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2af382ef 2021-08-11T17:21:37.754Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b4388089-ff26-43db-a740-8e56b060dcb6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:21:37.754Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b4388089-ff26-43db-a740-8e56b060dcb6 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:21:40.865Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b586d2d-590c-498c-8283-59493fcf3b01 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:21:40.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b586d2d-590c-498c-8283-59493fcf3b01 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5e8a9045 2021-08-11T17:21:40.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b586d2d-590c-498c-8283-59493fcf3b01 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:21:40.866Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2b586d2d-590c-498c-8283-59493fcf3b01 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:21:43.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c8ef5e5-9db1-4e96-9857-d07b792550e6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:21:43.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c8ef5e5-9db1-4e96-9857-d07b792550e6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@66da6c6f 2021-08-11T17:21:43.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c8ef5e5-9db1-4e96-9857-d07b792550e6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:21:43.597Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c8ef5e5-9db1-4e96-9857-d07b792550e6 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:21:46.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=91040ed7-c1ea-4704-a9c1-2abe031e580b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:21:46.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=91040ed7-c1ea-4704-a9c1-2abe031e580b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@9c9fe9f 2021-08-11T17:21:46.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=91040ed7-c1ea-4704-a9c1-2abe031e580b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:21:46.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=91040ed7-c1ea-4704-a9c1-2abe031e580b PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:21:48.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:21:48.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:21:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T17:21:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T17:21:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T17:21:49.949Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0cb2686-c3f7-460c-9ff3-0248e8b0c459 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:21:49.949Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0cb2686-c3f7-460c-9ff3-0248e8b0c459 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3f1bf1b3 2021-08-11T17:21:49.949Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0cb2686-c3f7-460c-9ff3-0248e8b0c459 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:21:49.949Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f0cb2686-c3f7-460c-9ff3-0248e8b0c459 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:21:53.148Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3af1fed2-48dd-4f8a-a685-44c77beaedc5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:21:53.148Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3af1fed2-48dd-4f8a-a685-44c77beaedc5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@351956ad 2021-08-11T17:21:53.148Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3af1fed2-48dd-4f8a-a685-44c77beaedc5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:21:53.148Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3af1fed2-48dd-4f8a-a685-44c77beaedc5 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:21:55.923Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=26056cc0-d096-4778-b36e-dfafb78412f6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:21:55.923Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=26056cc0-d096-4778-b36e-dfafb78412f6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@95ed75 2021-08-11T17:21:55.923Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=26056cc0-d096-4778-b36e-dfafb78412f6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:21:55.923Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=26056cc0-d096-4778-b36e-dfafb78412f6 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:21:58.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8fd74a3d-a559-4149-8369-706a9668bb93 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:21:58.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8fd74a3d-a559-4149-8369-706a9668bb93 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@466ec6fe 2021-08-11T17:21:58.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8fd74a3d-a559-4149-8369-706a9668bb93 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:21:58.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8fd74a3d-a559-4149-8369-706a9668bb93 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:22:01.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d3b7d22c-4b81-47cb-9805-fc72df4830aa PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:22:01.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d3b7d22c-4b81-47cb-9805-fc72df4830aa PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@269831e9 2021-08-11T17:22:01.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d3b7d22c-4b81-47cb-9805-fc72df4830aa PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:22:01.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d3b7d22c-4b81-47cb-9805-fc72df4830aa PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:22:03.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:22:03.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:22:04.254Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:22:04.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=234cc7e1-a1b5-4dcc-9a6d-afef595ee878 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:22:04.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=234cc7e1-a1b5-4dcc-9a6d-afef595ee878 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@594ba7a9 2021-08-11T17:22:04.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=234cc7e1-a1b5-4dcc-9a6d-afef595ee878 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:22:04.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=234cc7e1-a1b5-4dcc-9a6d-afef595ee878 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:22:07.609Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d553d52d-abeb-480c-906a-e9c82adf07da PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:22:07.609Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d553d52d-abeb-480c-906a-e9c82adf07da PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@769a17b0 2021-08-11T17:22:07.609Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d553d52d-abeb-480c-906a-e9c82adf07da PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:22:07.609Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d553d52d-abeb-480c-906a-e9c82adf07da PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:22:10.601Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5a4f881-affc-4ef8-aa5d-e3e8de82d4e5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:22:10.601Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5a4f881-affc-4ef8-aa5d-e3e8de82d4e5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@46f376ab 2021-08-11T17:22:10.601Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5a4f881-affc-4ef8-aa5d-e3e8de82d4e5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:22:10.601Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5a4f881-affc-4ef8-aa5d-e3e8de82d4e5 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:22:13.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab384f95-c3b0-45e9-a2fc-58f0c5d81abf PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:22:13.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab384f95-c3b0-45e9-a2fc-58f0c5d81abf PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@210310e5 2021-08-11T17:22:13.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab384f95-c3b0-45e9-a2fc-58f0c5d81abf PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:22:13.673Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=ab384f95-c3b0-45e9-a2fc-58f0c5d81abf PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:22:16.598Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a2fee699-9b75-43bb-8fb0-0492b7a4872d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:22:16.598Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a2fee699-9b75-43bb-8fb0-0492b7a4872d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@25719d83 2021-08-11T17:22:16.598Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a2fee699-9b75-43bb-8fb0-0492b7a4872d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:22:16.598Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a2fee699-9b75-43bb-8fb0-0492b7a4872d PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:22:18.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:22:18.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:22:19.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=83466652-1d6b-41ff-861f-3f6c60872f3e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:22:19.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=83466652-1d6b-41ff-861f-3f6c60872f3e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3f0390a 2021-08-11T17:22:19.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=83466652-1d6b-41ff-861f-3f6c60872f3e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:22:19.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=83466652-1d6b-41ff-861f-3f6c60872f3e PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:22:22.696Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10b1c07e-943e-4854-9d0a-65b7681bd3c4 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:22:22.696Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10b1c07e-943e-4854-9d0a-65b7681bd3c4 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5e6b51f1 2021-08-11T17:22:22.696Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10b1c07e-943e-4854-9d0a-65b7681bd3c4 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:22:22.696Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=10b1c07e-943e-4854-9d0a-65b7681bd3c4 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:22:25.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=84732130-5cc9-4b16-a026-3abb4dfe7553 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:22:25.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=84732130-5cc9-4b16-a026-3abb4dfe7553 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6756f6e1 2021-08-11T17:22:25.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=84732130-5cc9-4b16-a026-3abb4dfe7553 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:22:25.676Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=84732130-5cc9-4b16-a026-3abb4dfe7553 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:22:28.855Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dad7d1b6-fdc1-4f78-92b1-b7578d3a80da PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:22:28.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dad7d1b6-fdc1-4f78-92b1-b7578d3a80da PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4ee2e20d 2021-08-11T17:22:28.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dad7d1b6-fdc1-4f78-92b1-b7578d3a80da PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:22:28.856Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=dad7d1b6-fdc1-4f78-92b1-b7578d3a80da PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:22:31.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64e67631-5e26-442a-8ca4-af0741b76c9c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:22:31.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64e67631-5e26-442a-8ca4-af0741b76c9c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4c58bab2 2021-08-11T17:22:31.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64e67631-5e26-442a-8ca4-af0741b76c9c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:22:31.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=64e67631-5e26-442a-8ca4-af0741b76c9c PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:22:33.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:22:33.318Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:22:34.600Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72acf319-356e-4a4a-988f-bc6fef4deca1 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:22:34.600Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72acf319-356e-4a4a-988f-bc6fef4deca1 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@48389e11 2021-08-11T17:22:34.600Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72acf319-356e-4a4a-988f-bc6fef4deca1 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:22:34.600Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=72acf319-356e-4a4a-988f-bc6fef4deca1 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:22:37.600Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e7b1dd77-318f-402a-8e20-72bc621a4689 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:22:37.600Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e7b1dd77-318f-402a-8e20-72bc621a4689 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@18098809 2021-08-11T17:22:37.600Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e7b1dd77-318f-402a-8e20-72bc621a4689 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:22:37.600Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=e7b1dd77-318f-402a-8e20-72bc621a4689 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:22:40.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d27d54dd-67f3-4714-b0d1-b534c492807e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:22:40.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d27d54dd-67f3-4714-b0d1-b534c492807e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@41a2e38f 2021-08-11T17:22:40.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d27d54dd-67f3-4714-b0d1-b534c492807e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:22:40.659Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=d27d54dd-67f3-4714-b0d1-b534c492807e PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:22:43.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a0a65dad-8cb0-416a-9818-717e219d4b72 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:22:43.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a0a65dad-8cb0-416a-9818-717e219d4b72 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@48265c23 2021-08-11T17:22:43.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a0a65dad-8cb0-416a-9818-717e219d4b72 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:22:43.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a0a65dad-8cb0-416a-9818-717e219d4b72 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:22:46.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=161eb079-b245-4b21-b756-b93bce9561de PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:22:46.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=161eb079-b245-4b21-b756-b93bce9561de PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@44629b20 2021-08-11T17:22:46.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=161eb079-b245-4b21-b756-b93bce9561de PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:22:46.616Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=161eb079-b245-4b21-b756-b93bce9561de PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:22:48.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:22:48.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:22:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T17:22:49.291Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T17:22:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T17:22:50.550Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=230ccf18-285d-4320-9249-a283a0b9710d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:22:50.550Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=230ccf18-285d-4320-9249-a283a0b9710d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@10ff1a65 2021-08-11T17:22:50.550Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=230ccf18-285d-4320-9249-a283a0b9710d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:22:50.550Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=230ccf18-285d-4320-9249-a283a0b9710d PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:22:53.262Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7816b97d-6f66-427d-a2af-a7ce9657bbff PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:22:53.262Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7816b97d-6f66-427d-a2af-a7ce9657bbff PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@46972d6e 2021-08-11T17:22:53.262Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7816b97d-6f66-427d-a2af-a7ce9657bbff PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:22:53.262Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7816b97d-6f66-427d-a2af-a7ce9657bbff PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:22:56.852Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25e9972b-625e-4e2c-8040-c96288f82c5e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:22:56.852Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25e9972b-625e-4e2c-8040-c96288f82c5e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3caaa6db 2021-08-11T17:22:56.852Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25e9972b-625e-4e2c-8040-c96288f82c5e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:22:56.852Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=25e9972b-625e-4e2c-8040-c96288f82c5e PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:22:59.751Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68d658bb-f7f2-4145-bbe7-fbb54b7aa89b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:22:59.751Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68d658bb-f7f2-4145-bbe7-fbb54b7aa89b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6c4191f5 2021-08-11T17:22:59.751Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68d658bb-f7f2-4145-bbe7-fbb54b7aa89b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:22:59.751Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=68d658bb-f7f2-4145-bbe7-fbb54b7aa89b PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:23:01.984Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bbb11160-66d3-4dcf-9e4c-5cf3cd837075 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:23:01.984Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bbb11160-66d3-4dcf-9e4c-5cf3cd837075 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@405922a 2021-08-11T17:23:01.984Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bbb11160-66d3-4dcf-9e4c-5cf3cd837075 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:23:01.984Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bbb11160-66d3-4dcf-9e4c-5cf3cd837075 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:23:03.306Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:23:03.312Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:23:04.248Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:23:04.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5737461-b1f3-4aef-b18a-ca68d556fcf9 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:23:04.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5737461-b1f3-4aef-b18a-ca68d556fcf9 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7b062399 2021-08-11T17:23:04.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5737461-b1f3-4aef-b18a-ca68d556fcf9 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:23:04.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b5737461-b1f3-4aef-b18a-ca68d556fcf9 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:23:07.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a040415d-d03b-4bfe-9b69-5d4c53903efe PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:23:07.652Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a040415d-d03b-4bfe-9b69-5d4c53903efe PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1273ce5c 2021-08-11T17:23:07.652Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a040415d-d03b-4bfe-9b69-5d4c53903efe PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:23:07.652Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=a040415d-d03b-4bfe-9b69-5d4c53903efe PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:23:10.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f5233f42-dcc4-4acd-88b6-a4e5f5b23922 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:23:10.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f5233f42-dcc4-4acd-88b6-a4e5f5b23922 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@725d2bde 2021-08-11T17:23:10.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f5233f42-dcc4-4acd-88b6-a4e5f5b23922 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:23:10.607Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f5233f42-dcc4-4acd-88b6-a4e5f5b23922 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:23:13.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41c7cd78-009e-422f-988c-6d892e30e714 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:23:13.963Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41c7cd78-009e-422f-988c-6d892e30e714 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5a61737d 2021-08-11T17:23:13.964Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41c7cd78-009e-422f-988c-6d892e30e714 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:23:13.964Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=41c7cd78-009e-422f-988c-6d892e30e714 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:23:16.594Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73d6efed-2ab4-4607-9671-55b660dc72e3 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:23:16.595Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73d6efed-2ab4-4607-9671-55b660dc72e3 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@10319943 2021-08-11T17:23:16.595Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73d6efed-2ab4-4607-9671-55b660dc72e3 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:23:16.595Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73d6efed-2ab4-4607-9671-55b660dc72e3 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:23:18.305Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:23:18.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:23:19.612Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b34c7d0c-7239-4216-a9e6-ba032de80f37 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:23:19.612Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b34c7d0c-7239-4216-a9e6-ba032de80f37 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1c3ca437 2021-08-11T17:23:19.612Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b34c7d0c-7239-4216-a9e6-ba032de80f37 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:23:19.612Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b34c7d0c-7239-4216-a9e6-ba032de80f37 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:23:22.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0c8c763-55d4-4f50-900d-2778e2b72b81 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:23:22.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0c8c763-55d4-4f50-900d-2778e2b72b81 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7a46fb5 2021-08-11T17:23:22.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0c8c763-55d4-4f50-900d-2778e2b72b81 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:23:22.668Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c0c8c763-55d4-4f50-900d-2778e2b72b81 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:23:25.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55f2218a-2e46-4e0b-9753-4cfd859c316d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:23:25.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55f2218a-2e46-4e0b-9753-4cfd859c316d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@568f0ea7 2021-08-11T17:23:25.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55f2218a-2e46-4e0b-9753-4cfd859c316d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:23:25.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=55f2218a-2e46-4e0b-9753-4cfd859c316d PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:23:28.849Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=791e43a7-0c7b-4bb3-9f34-578dbb655f58 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:23:28.849Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=791e43a7-0c7b-4bb3-9f34-578dbb655f58 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4a686eb4 2021-08-11T17:23:28.849Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=791e43a7-0c7b-4bb3-9f34-578dbb655f58 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:23:28.849Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=791e43a7-0c7b-4bb3-9f34-578dbb655f58 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:23:31.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=599a2acf-86b1-40ef-86e3-bbddd2d1fe9b PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:23:31.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=599a2acf-86b1-40ef-86e3-bbddd2d1fe9b PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@330c76d2 2021-08-11T17:23:31.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=599a2acf-86b1-40ef-86e3-bbddd2d1fe9b PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:23:31.679Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=599a2acf-86b1-40ef-86e3-bbddd2d1fe9b PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:23:33.305Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:23:33.309Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:23:34.648Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f142660e-588a-4a82-9563-81d5a5106179 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:23:34.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f142660e-588a-4a82-9563-81d5a5106179 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@17f17ec1 2021-08-11T17:23:34.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f142660e-588a-4a82-9563-81d5a5106179 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:23:34.649Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f142660e-588a-4a82-9563-81d5a5106179 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:23:37.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1d25cce-a3a9-4175-a014-729e6dad2684 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:23:37.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1d25cce-a3a9-4175-a014-729e6dad2684 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@39a5a0ae 2021-08-11T17:23:37.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1d25cce-a3a9-4175-a014-729e6dad2684 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:23:37.681Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f1d25cce-a3a9-4175-a014-729e6dad2684 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:23:40.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e078f2d-193e-4c6a-810e-18cf70b49011 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:23:40.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e078f2d-193e-4c6a-810e-18cf70b49011 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@303174e0 2021-08-11T17:23:40.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e078f2d-193e-4c6a-810e-18cf70b49011 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:23:40.859Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4e078f2d-193e-4c6a-810e-18cf70b49011 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:23:43.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50bce5dc-4673-4bd7-a8f2-4d5ef88d60a6 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:23:43.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50bce5dc-4673-4bd7-a8f2-4d5ef88d60a6 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@605b3ed2 2021-08-11T17:23:43.614Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50bce5dc-4673-4bd7-a8f2-4d5ef88d60a6 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:23:43.615Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50bce5dc-4673-4bd7-a8f2-4d5ef88d60a6 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:23:46.599Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bef5f1f5-2fc9-4084-a29a-aa7eb0ce0f54 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:23:46.599Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bef5f1f5-2fc9-4084-a29a-aa7eb0ce0f54 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@7c40436e 2021-08-11T17:23:46.599Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bef5f1f5-2fc9-4084-a29a-aa7eb0ce0f54 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:23:46.599Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=bef5f1f5-2fc9-4084-a29a-aa7eb0ce0f54 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:23:48.305Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:23:48.310Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:23:49.290Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T17:23:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T17:23:49.291Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T17:23:50.149Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=20091e77-8ad7-4375-be5b-a270854e5f0e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:23:50.149Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=20091e77-8ad7-4375-be5b-a270854e5f0e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@68f43c98 2021-08-11T17:23:50.149Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=20091e77-8ad7-4375-be5b-a270854e5f0e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:23:50.149Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=20091e77-8ad7-4375-be5b-a270854e5f0e PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:23:53.049Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c4d37273-fd65-49d2-a4dc-2f3363e00ace PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:23:53.049Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c4d37273-fd65-49d2-a4dc-2f3363e00ace PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5359d312 2021-08-11T17:23:53.049Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c4d37273-fd65-49d2-a4dc-2f3363e00ace PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:23:53.049Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c4d37273-fd65-49d2-a4dc-2f3363e00ace PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:23:55.694Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fc5f27d9-e269-46dc-9720-2071bbbca894 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:23:55.694Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fc5f27d9-e269-46dc-9720-2071bbbca894 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6a4dda05 2021-08-11T17:23:55.694Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fc5f27d9-e269-46dc-9720-2071bbbca894 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:23:55.694Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=fc5f27d9-e269-46dc-9720-2071bbbca894 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:23:58.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1cdf1fd2-505e-41d1-adf3-e1aa3db81480 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:23:58.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1cdf1fd2-505e-41d1-adf3-e1aa3db81480 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1468af11 2021-08-11T17:23:58.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1cdf1fd2-505e-41d1-adf3-e1aa3db81480 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:23:58.647Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=1cdf1fd2-505e-41d1-adf3-e1aa3db81480 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:24:01.590Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c177e77-2953-43bc-aa38-0e16492e7e08 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:24:01.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c177e77-2953-43bc-aa38-0e16492e7e08 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5db151f8 2021-08-11T17:24:01.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c177e77-2953-43bc-aa38-0e16492e7e08 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:24:01.591Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4c177e77-2953-43bc-aa38-0e16492e7e08 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:24:03.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:24:03.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:24:04.261Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:24:04.658Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4deef7b2-5da0-424d-bdca-49fd72803b89 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:24:04.658Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4deef7b2-5da0-424d-bdca-49fd72803b89 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@40edd81f 2021-08-11T17:24:04.658Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4deef7b2-5da0-424d-bdca-49fd72803b89 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:24:04.658Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4deef7b2-5da0-424d-bdca-49fd72803b89 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:24:07.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4af44521-886d-47b0-bcdd-dc9caa508f52 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:24:07.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4af44521-886d-47b0-bcdd-dc9caa508f52 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@44f4aff5 2021-08-11T17:24:07.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4af44521-886d-47b0-bcdd-dc9caa508f52 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:24:07.685Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=4af44521-886d-47b0-bcdd-dc9caa508f52 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:24:10.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0dfbcdd4-5365-4d8c-a361-3c106c7678c0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:24:10.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0dfbcdd4-5365-4d8c-a361-3c106c7678c0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1535905d 2021-08-11T17:24:10.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0dfbcdd4-5365-4d8c-a361-3c106c7678c0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:24:10.593Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0dfbcdd4-5365-4d8c-a361-3c106c7678c0 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:24:13.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12e56d76-9d1c-46b3-9113-dbcab1fa19f5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:24:13.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12e56d76-9d1c-46b3-9113-dbcab1fa19f5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@42dc2d92 2021-08-11T17:24:13.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12e56d76-9d1c-46b3-9113-dbcab1fa19f5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:24:13.621Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=12e56d76-9d1c-46b3-9113-dbcab1fa19f5 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:24:16.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50743115-5ee7-40f4-9480-91b4eca4606d PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:24:16.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50743115-5ee7-40f4-9480-91b4eca4606d PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6c5f28f6 2021-08-11T17:24:16.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50743115-5ee7-40f4-9480-91b4eca4606d PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:24:16.683Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=50743115-5ee7-40f4-9480-91b4eca4606d PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:24:18.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:24:18.314Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:24:19.707Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47ddb1bb-061f-4781-9db6-6be995aebfde PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:24:19.707Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47ddb1bb-061f-4781-9db6-6be995aebfde PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@d35b98f 2021-08-11T17:24:19.707Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47ddb1bb-061f-4781-9db6-6be995aebfde PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:24:19.707Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47ddb1bb-061f-4781-9db6-6be995aebfde PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:24:22.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f5a6f826-01ca-4f62-8acf-a7f3584aade8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:24:22.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f5a6f826-01ca-4f62-8acf-a7f3584aade8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@34bb770c 2021-08-11T17:24:22.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f5a6f826-01ca-4f62-8acf-a7f3584aade8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:24:22.667Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f5a6f826-01ca-4f62-8acf-a7f3584aade8 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:24:25.669Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2908eed5-fc60-45b5-963f-c22f6462c3a2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:24:25.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2908eed5-fc60-45b5-963f-c22f6462c3a2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@17a87cb4 2021-08-11T17:24:25.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2908eed5-fc60-45b5-963f-c22f6462c3a2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:24:25.670Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2908eed5-fc60-45b5-963f-c22f6462c3a2 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:24:28.863Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4f73907-e157-499f-92e9-ffe1664f4447 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:24:28.863Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4f73907-e157-499f-92e9-ffe1664f4447 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@2a5ee2f 2021-08-11T17:24:28.863Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4f73907-e157-499f-92e9-ffe1664f4447 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:24:28.863Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f4f73907-e157-499f-92e9-ffe1664f4447 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:24:31.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73777e8b-c6d0-4a6f-85e8-c4ed22a32b12 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:24:31.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73777e8b-c6d0-4a6f-85e8-c4ed22a32b12 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@22fcc635 2021-08-11T17:24:31.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73777e8b-c6d0-4a6f-85e8-c4ed22a32b12 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:24:31.664Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=73777e8b-c6d0-4a6f-85e8-c4ed22a32b12 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:24:33.307Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:24:33.315Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:24:34.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9247e051-c766-4587-b864-45e98ebe5683 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:24:34.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9247e051-c766-4587-b864-45e98ebe5683 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4d10d671 2021-08-11T17:24:34.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9247e051-c766-4587-b864-45e98ebe5683 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:24:34.620Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=9247e051-c766-4587-b864-45e98ebe5683 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:24:37.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=62bb9bcb-f704-4555-8b57-0b6d36892bd5 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:24:37.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=62bb9bcb-f704-4555-8b57-0b6d36892bd5 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@19895ebb 2021-08-11T17:24:37.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=62bb9bcb-f704-4555-8b57-0b6d36892bd5 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:24:37.651Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=62bb9bcb-f704-4555-8b57-0b6d36892bd5 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:24:40.612Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47e4724f-12c6-487d-8204-222ec0c41b42 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:24:40.612Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47e4724f-12c6-487d-8204-222ec0c41b42 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@34f01f33 2021-08-11T17:24:40.612Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47e4724f-12c6-487d-8204-222ec0c41b42 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:24:40.612Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=47e4724f-12c6-487d-8204-222ec0c41b42 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:24:43.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f16c6b2-8089-46d0-aa1e-fe9d00b68efe PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:24:43.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f16c6b2-8089-46d0-aa1e-fe9d00b68efe PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@70f9aaf6 2021-08-11T17:24:43.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f16c6b2-8089-46d0-aa1e-fe9d00b68efe PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:24:43.592Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3f16c6b2-8089-46d0-aa1e-fe9d00b68efe PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:24:46.601Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=65720623-9ce2-42c4-8e88-2842ec8f9873 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:24:46.601Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=65720623-9ce2-42c4-8e88-2842ec8f9873 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@3b1159c6 2021-08-11T17:24:46.601Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=65720623-9ce2-42c4-8e88-2842ec8f9873 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:24:46.601Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=65720623-9ce2-42c4-8e88-2842ec8f9873 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:24:48.308Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:24:48.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:24:49.289Z [TopicPollingThread-1] INFO c.a.n.c.c.impl.CambriaConsumerImpl UEB GET /events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 2021-08-11T17:24:49.290Z [TopicPollingThread-1] WARN c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 will send credentials over a clear channel. 2021-08-11T17:24:49.290Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/events/SDC-DISTR-STATUS-TOPIC-AUTO/sdc-AUTO/sdc-AUTO1?timeout=15000 (as iPIxkpAMI8qTcQj8) ... 2021-08-11T17:24:50.455Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c6f2776-ae84-4916-825e-4e50e054d57c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:24:50.455Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c6f2776-ae84-4916-825e-4e50e054d57c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@683ea499 2021-08-11T17:24:50.455Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c6f2776-ae84-4916-825e-4e50e054d57c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:24:50.455Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=2c6f2776-ae84-4916-825e-4e50e054d57c PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:24:53.348Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b9e48efe-cdfc-4da1-b190-8f8e4ca5bc6e PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:24:53.348Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b9e48efe-cdfc-4da1-b190-8f8e4ca5bc6e PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@5c03272a 2021-08-11T17:24:53.348Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b9e48efe-cdfc-4da1-b190-8f8e4ca5bc6e PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:24:53.348Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=b9e48efe-cdfc-4da1-b190-8f8e4ca5bc6e PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:24:56.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82dbcc86-c4af-4079-ac25-046e688ec903 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:24:56.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82dbcc86-c4af-4079-ac25-046e688ec903 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@61109846 2021-08-11T17:24:56.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82dbcc86-c4af-4079-ac25-046e688ec903 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:24:56.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=82dbcc86-c4af-4079-ac25-046e688ec903 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:24:59.258Z [cluster1-nio-worker-1] WARN c.d.driver.core.HostConnectionPool Error creating connection to cassandra.onap/10.242.184.131:9042 com.datastax.driver.core.exceptions.TransportException: [cassandra.onap/10.242.184.131:9042] Connection has been closed at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1427) at com.datastax.driver.core.Connection$ConnectionCloseFuture.force(Connection.java:1408) at com.datastax.driver.core.Connection.defunct(Connection.java:585) at com.datastax.driver.core.Connection$Dispatcher.exceptionCaught(Connection.java:1330) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273) at io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.handleReadException(AbstractEpollStreamChannel.java:728) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:818) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Unknown Source) 2021-08-11T17:24:59.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3838f93f-5832-4843-b8ce-d79f4e6ba2e8 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:24:59.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3838f93f-5832-4843-b8ce-d79f4e6ba2e8 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@507e32cb 2021-08-11T17:24:59.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3838f93f-5832-4843-b8ce-d79f4e6ba2e8 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:24:59.258Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=3838f93f-5832-4843-b8ce-d79f4e6ba2e8 PartnerName=SDC.BE The cassandra down nodes number is 1 2021-08-11T17:24:59.568Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=3838f93f-5832-4843-b8ce-d79f4e6ba2e8 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component ON_BOARDING state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T17:24:59.568Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=3838f93f-5832-4843-b8ce-d79f4e6ba2e8 ErrorCategory=INFO ServiceName=BE Component ON_BOARDING state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T17:25:01.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d914c7d-60f8-4d4b-8c76-00c59da9fba2 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:25:01.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d914c7d-60f8-4d4b-8c76-00c59da9fba2 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4df94e31 2021-08-11T17:25:01.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d914c7d-60f8-4d4b-8c76-00c59da9fba2 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:25:01.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=7d914c7d-60f8-4d4b-8c76-00c59da9fba2 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:25:01.600Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=7d914c7d-60f8-4d4b-8c76-00c59da9fba2 ErrorCategory=INFO ServiceName=BE Component CASSANDRA state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component CASSANDRA state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T17:25:01.600Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=7d914c7d-60f8-4d4b-8c76-00c59da9fba2 ErrorCategory=ERROR ServiceName=BE Component CASSANDRA state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-08-11T17:25:02.248Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:02.300Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:02.403Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:02.506Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:02.609Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:02.812Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:03.015Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:03.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=INFO ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:25:03.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck ASDC Backend probably lost connectivity to JanusGraph DB. Please check the logs for more information. 2021-08-11T17:25:03.305Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:25:03.310Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:25:03.418Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:03.821Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:04.224Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:04.247Z [TopicPollingThread-1] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:25:04.577Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=583d5597-de36-439c-8c8d-6ce5ce5baa47 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:25:04.577Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=583d5597-de36-439c-8c8d-6ce5ce5baa47 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@8861ad3 2021-08-11T17:25:04.577Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=583d5597-de36-439c-8c8d-6ce5ce5baa47 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:25:04.577Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=583d5597-de36-439c-8c8d-6ce5ce5baa47 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:25:04.591Z [BE-Health-Check-Task] INFO o.o.s.c.config.EcompErrorLogUtil alarmSeverity=INFORMATIONAL RequestID=583d5597-de36-439c-8c8d-6ce5ce5baa47 ErrorCategory=INFO ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=BE Component JANUSGRAPH state changed from UP to DOWN ASDC Backend Recovery to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. 2021-08-11T17:25:04.591Z [BE-Health-Check-Task] ERROR o.o.s.c.config.EcompErrorLogUtil alarmSeverity=CRITICAL RequestID=583d5597-de36-439c-8c8d-6ce5ce5baa47 ErrorCategory=ERROR ServiceName=BE Component JANUSGRAPH state changed from UP to DOWN TargetServiceName=null ErrorCode=200 PartnerName=SDC.BE TargetEntity=beHealthCheck ASDC Backend probably lost connectivity to either one of the following components: JanusGraph DB, Cassandra, Onboarding, UEB Cluster. Please check the logs for more information. 2021-08-11T17:25:05.027Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:05.665Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:05.717Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:05.820Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:05.829Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:06.022Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:06.225Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:06.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:25:06.648Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:06.648Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:06.851Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:07.054Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:07.456Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:07.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=97cea3e3-9d84-4bb5-91f1-2cd0baa2fd99 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:25:07.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=97cea3e3-9d84-4bb5-91f1-2cd0baa2fd99 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@6d31712d 2021-08-11T17:25:07.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=97cea3e3-9d84-4bb5-91f1-2cd0baa2fd99 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:25:07.581Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=97cea3e3-9d84-4bb5-91f1-2cd0baa2fd99 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:25:07.859Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:08.251Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:08.261Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:09.064Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T17:25:00Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:09.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:25:10.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6562f381-dee1-4f74-8e95-46a041d0560c PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:25:10.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6562f381-dee1-4f74-8e95-46a041d0560c PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4c78ee2a 2021-08-11T17:25:10.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6562f381-dee1-4f74-8e95-46a041d0560c PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:25:10.578Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=6562f381-dee1-4f74-8e95-46a041d0560c PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:25:11.456Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:11.509Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:11.562Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:11.614Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:11.667Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:11.773Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:11.976Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:12.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:25:12.379Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:13.182Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:13.574Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8fc09460-d9dd-4ab8-aa83-56bfdfc68460 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:25:13.574Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8fc09460-d9dd-4ab8-aa83-56bfdfc68460 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@4bf7a952 2021-08-11T17:25:13.574Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8fc09460-d9dd-4ab8-aa83-56bfdfc68460 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:25:13.575Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=8fc09460-d9dd-4ab8-aa83-56bfdfc68460 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:25:14.066Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:14.094Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:14.120Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:14.147Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:14.174Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:14.227Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:14.329Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:14.432Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:14.634Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:14.784Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:14.837Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:15.239Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:15.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:25:16.042Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:16.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0213f492-8d13-4303-b4f1-3da5f172dfe0 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:25:16.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0213f492-8d13-4303-b4f1-3da5f172dfe0 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@731f727d 2021-08-11T17:25:16.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0213f492-8d13-4303-b4f1-3da5f172dfe0 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:25:16.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=0213f492-8d13-4303-b4f1-3da5f172dfe0 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:25:17.656Z [pool-8-thread-1] WARN o.j.diskstorage.log.kcvs.KCVSLog Could not read messages for timestamp [2021-08-11T17:25:00Z] (this read will be retried) org.janusgraph.core.JanusGraphException: Could not execute operation due to backend exception at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:56) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Could not successfully complete backend operation due to repeated temporary exceptions after PT4S at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:100) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) ... 8 common frames omitted Caused by: org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) ... 9 common frames omitted Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:17.987Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:18.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:25:18.305Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient GET http://message-router.onap:3904/apiKeys/iPIxkpAMI8qTcQj8 (anonymous) ... 2021-08-11T17:25:18.313Z [UEB-Health-Check-Thread] INFO c.att.nsa.apiClient.http.HttpClient \t--> HTTP/1.1 200 OK 2021-08-11T17:25:19.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f20ab0ac-e688-4e07-94d2-be55b3f0f3cc PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:25:19.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f20ab0ac-e688-4e07-94d2-be55b3f0f3cc PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@15c1cd2a 2021-08-11T17:25:19.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f20ab0ac-e688-4e07-94d2-be55b3f0f3cc PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:25:19.580Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=f20ab0ac-e688-4e07-94d2-be55b3f0f3cc PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:25:21.191Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:21.218Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:21.244Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:21.247Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:25:21.296Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:21.348Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:21.401Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:21.503Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:21.706Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:21.910Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:22.113Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:22.515Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:22.587Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c9f5ed5b-52d3-4a5f-99b0-f7256b9cc127 PartnerName=SDC.BE creating cluster for Cassandra for monitoring. 2021-08-11T17:25:22.587Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c9f5ed5b-52d3-4a5f-99b0-f7256b9cc127 PartnerName=SDC.BE The cassandra session is com.datastax.driver.core.SessionManager@1e436806 2021-08-11T17:25:22.587Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c9f5ed5b-52d3-4a5f-99b0-f7256b9cc127 PartnerName=SDC.BE The number of cassandra nodes is:3 2021-08-11T17:25:22.587Z [BE-Health-Check-Task] INFO o.o.s.b.c.impl.CassandraHealthCheck RequestID=c9f5ed5b-52d3-4a5f-99b0-f7256b9cc127 PartnerName=SDC.BE The cassandra down nodes number is 2 2021-08-11T17:25:22.659Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:22.711Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:22.814Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:23.017Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:23.318Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:23.420Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:23.823Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:24.226Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:24.244Z [JanusGraph-Health-Check-Task] ERROR o.o.s.b.d.j.JanusGraphClient RequestID=null ErrorCategory=ERROR ServerIPAddress=10.242.24.223 ServiceName=janusgraphHealthCheck TargetServiceName=null ErrorCode=200 ServerFQDN=dev-sdc-be-997dd74c7-dmm7p TargetEntity=janusgraphHealthCheck JanusGraph Health Check Failed. java.util.concurrent.TimeoutException 2021-08-11T17:25:24.921Z [JanusGraph-Health-Check-Thread] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [VertexIndexQuery]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.keycolumnvalue.KCVSProxy.getSlice(KCVSProxy.java:77) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:399) at org.janusgraph.diskstorage.BackendTransaction$5.call(BackendTransaction.java:396) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470) at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:396) at org.janusgraph.graphdb.query.graph.MultiKeySliceQuery.execute(MultiKeySliceQuery.java:51) at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:512) at org.janusgraph.graphdb.util.SubqueryIterator.(SubqueryIterator.java:66) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295) at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190) at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194) at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68) at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54) at org.janusgraph.graphdb.query.ResultSetIterator.(ResultSetIterator.java:44) at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66) at com.google.common.collect.Iterables$4.iterator(Iterables.java:559) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:340) at org.openecomp.sdc.be.dao.janusgraph.JanusGraphClient$HealthCheckTask.call(JanusGraphClient.java:336) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ... 4 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency LOCAL_QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted 2021-08-11T17:25:25.029Z [pool-8-thread-1] INFO o.j.d.util.BackendOperation Temporary exception during backend operation [messageReading@0:0]. Attempting backoff retry. org.janusgraph.diskstorage.TemporaryBackendException: Temporary failure in storage backend at io.vavr.API$Match$Case0.apply(API.java:3174) at io.vavr.API$Match.of(API.java:3137) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.lambda$static$0(CQLKeyColumnValueStore.java:123) at io.vavr.control.Try.getOrElseThrow(Try.java:671) at org.janusgraph.diskstorage.cql.CQLKeyColumnValueStore.getSlice(CQLKeyColumnValueStore.java:290) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:792) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller$1.call(KCVSLog.java:789) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:147) at org.janusgraph.diskstorage.util.BackendOperation$1.call(BackendOperation.java:161) at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54) at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:158) at org.janusgraph.diskstorage.log.kcvs.KCVSLog$MessagePuller.run(KCVSLog.java:726) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.util.concurrent.ExecutionException: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566) at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547) at io.vavr.control.Try.of(Try.java:62) at io.vavr.concurrent.FutureImpl.lambda$run$2(FutureImpl.java:199) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ... 3 common frames omitted Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /10.242.154.72:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency QUORUM (2 required but only 1 alive))) at com.datastax.driver.core.RequestHandler.reportNoMoreHosts(RequestHandler.java:283) at com.datastax.driver.core.RequestHandler.access$1200(RequestHandler.java:61) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.findNextHostAndQuery(RequestHandler.java:375) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.retry(RequestHandler.java:557) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.processRetryDecision(RequestHandler.java:539) at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:810) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1262) at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1180) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 common frames omitted